So why I have been incommunicado lately, or as @Unblakeable puts it, apparently in the witness protection program?  We’ve been heads down working on the next release of our platform, and launching several new customers (including a Global Fortune 100 customer – whee!).  Oh, and talking with investors, growing our base of channel partners, and pulling more than a few all-nighters. Suffice it to say that “free time” is somewhat of a myth around our office these days.

At any rate, I thought I would take a few minutes to share some of the new stuff that is in StarPound v1.3.  This release represents a tremendous step forward in the evolution of our platform, and ushers in a promising new phase for our company. After months of intense development, we firmly believe that this release represents the most advanced open-source Customer Interaction Management platform in the world. It is one that remains unique in it’s blend of business process management with advanced call center functionality.

Think Avaya – but think free. Yup.  Skills-based routing, cost-based routing, work queues, multi-channel communications, etc. That’s where we’re going, and to a very large extent, we are there with the 1.5 million lines of code in this release. Right now, it looks like we should be finished up and ready to release this version in the next week or so.

Some of the key features, changes, and other highlights of the v1.3 release are described below:

Greatly Enhanced Stability and Scalability

One of the areas we’ve really been focused on over the past 12 months is the “enterprise readiness” of our platform. We have spent quite a bit of time, energy, and money dedicated to the stress testing of the platform. Since v1.2 we’ve addressed over 200 defects, and have performed hundreds of hours of stress testing under a variety of different configurations and conditions. Particular emphasis was also placed upon the horizontal scalability of the StarPound platform.  Throughout this entire process, we’ve been iterating the platform, getting feedback, and testing it, in a live production environment with hundreds of agents servicing several very large Fortune customers.  The net result is the most stable and scalable release of the StarPound platform to-date.

Full Support for FreeSwitch

When we released StarPound CORE v1.2 last year, one of the key new features was our initial support for FreeSWITCH. If you aren’t familiar with FreeSWITCH, it is a pretty advanced open-source telephony platform for call switching, PBX, and media serving functionality, which competes directly with Asterisk. Well, in StarPound v1.3, we’ve expanded our support for FreeSWITCH.

In this release, we’ve replaced the original SIP-based call-control with native socket-based call-control using FreeSWITCH’s native command set. The result is vastly improved performance, and reduced consumption of system resources.

Telephony/Switching Abstraction Layer

Dovetailing on what was described above for FreeSWITCH, we’ve rearchitected the way that StarPound CORE communicates with backend telecom switches. As our desire is to be as agnostic as possible, we’ve implemented a Telephony Abstraction Layer that provides the interface model for call-control, media serving, etc. This version of StarPound will be distributed with three built-in connectors: generic SIP, Asterisk, and FreeSWITCH. Given the new architecture, it should make it easier to add connectors for other platforms down the road (we’re already in talks with certain commercial vendors about creating connectors for their switches as well).

Interval Metric Reporting

One of the cool new features of StarPound|Call Center v1.3 is an interval reporting capability. On a per-organization basis, the system collects and stores various data points, such as call detail activity, agent work activity, and agent status. This data now rolls up into our real-time dashboard view, where StarPound can calculate and report on things such as agent efficiency and average call hold time over each interval.

New StarPound API

As part of v1.3 of StarPound, we have introduced the first phase of our new API. Currently, the API supports about 60 web service calls (SOAP or REST) into StarPound PBX, and provides full access into everything from voicemails and extension management to custom ring plans (ring multiple devices, find me/follow me, a la Google-Voice, etc) for users. We will be further extending the API down the road to provide additional functionality for the API, including hooks into the call center platform.

Revised Documentation

As with most open source projects, our documentation always seems to lag behind the development of the codebase itself. We’ve spent quite a bit of effort in trying to bring our documentation up to snuff with the latest features and functionality of the platform. We’ve also moved our documentation from Word/PDF format into the web, which should make searching much easier.

And speaking of new web site – our new one, which will launch at the same time, provides a lot more in the way of fostering the growing community around our platform.  Forums, trouble tickets, FAQs, searchable documentation, how-to guides, tutorials, videos, etc.  Good stuff.

StyroFone

One of the coolest new things we’ve been working on is codenamed “StyroFone”.  There have only been a lucky few who have seen this thing in action. Unfortunately, we’re not quite at the point where I’m ready to show it off, but trust me when I say that it is very cool.  I will tell you this, though: I think it has the potential to significantly depreciate a lot of the value from certain PBX and telco providers. Stay tuned to styrofone.com for details as they become available.

A while back, I was approached by a team of entrepreneurs here in Atlanta who were in the planning phases of a new wireless startup called WiFi Cat.  Strange name, I know, but they had a pretty novel idea.  As I tend to do with so many startups these days, I offered my advice on how they could get the venture off the ground from a bootstrapping standpoint.  My advice, of course, is/was worth what they paid for it – zero lol.

At any rate, fast forward.  The team has really made a lot of progress with their model and service, and are finally ready to stand in front of investors and share their story.  I encouraged them to apply to StartupRiot (and CapitalLounge) to start getting their message out there.  They were accepted into both, and will be having their big coming out party at StartupRiot on February 18th here in Atlanta.

I am also thrilled, and flattered, that they asked me to serve on their board of directors, a responsibility that I do not often accept, and one that I take very seriously.  In the case of WiFi Cat, it was really a no-brainer, though.  This technology is going to fundamentally change the WiFi scene, with far reaching implications that will absolutely DWARF the WiMax initiative.

The team also asked if I could present the idea at StartupRiot personally.  While I don’t think I am necessarily the best candidate to pitch the idea (I’d prefer the founders), I think my passion around their idea will serve me well during the presentation.

So, I will see everyone on the 18th at StartupRiot!  Can’t wait until I can finally talk about this deal – and no, i can’t talk about it right now, so please don’t ask!

Cheers.

First, if  you aren’t familiar with Asterisk, it is the leading open-source software PBX and telephony package out in the marketplace.  They’ve done a remarkable job in creating something of enormous value.  I will go so far as to say I think Asterisk has quickly become an integral part of the open source enterprise stack.  Asterisk was created by Mark Spencer of Digium fame.

Asterisk is a telephony engine and toolkit.  Meaning you can use it as the underpinning of lots of really cool telephony apps.  You can use it as a PBX, a gateway, a media server, and even in call center contexts.

StarPound, while also an engine and toolkit, builds on top of toolkits like Asterisk.  StarPound consists of a visual business process modeling tool, and a suite of application servers that allow you to automate those processes.  For those processes that need to be “communication-enabled” (see: CEBP at Wikipedia), StarPound’s platform has telephony-related tools that tie into components at the Asterisk level.

StarPound relies upon external telephony engines like Asterisk to provide under-the-hood call control and media serving functionality.  Note:  We’ve done in-house integration and testing of FreeSwitch, and will be rolling out production support for that platform soon.

On top of all of these components, we have developed a suite of enterprise applications specifically aimed at call centers and enterprise PBX users.  These applications are incredible on their own merit, but also serve as great examples of the types of applications that can be built with the StarPound platform.  A lot of the confusion comes from people who equate StarPound as a “PBX in the cloud” company solely.  They think “PBX in the cloud”, and then immediately think Asterisk.

Even though we have a cloud-based PBX application, we typically don’t host it for individual companies.  That isn’t our model – that is the model of Vocalocity and others.  We are the type of company that powers another company that wants to be in that space.  We’ll have some big announcements to make soon on that …

So the short answer is, we are built on top of telephony toolkits like Asterisk and FreeSwitch, but that is really only a small portion of our overall capabilities.  StarPound is more accurately described as a software platform that automates business processes by turning them into web and voice services.  In fact, our PBX and call center applications are driven by, guess what?  Business process models automated through StarPound!  If you want to change something in your PBX or call center, you don’t fiddle with config files – you visually edit the “way” the application is supposed to work – the process model.

A great way to describe it  … with StarPound, you model what an application is supposed to do, not necessarily how it is supposed to do it.

Cheers.

As a FOSS (Free, Open Source Solution) company, that also offers a cloud-based software-as-a-service option, we’re sorting through more deal opportunities than we can handle right now. We’re hiring based upon real growth … which is the ultimate barometer of any startup’s progression.

“A down market is a great time for an emerging company to secure a beachhead against established players.”

CIOs and other tech decision makers still have the same problems to solve within their organizations, they just don’t have a blank check book to work with anymore.  No one ever got fired for bringing in a Microsoft, Avaya, SAP, or any other market leader to implement a solution.  But if they can’t afford to do that, they can either look to a startup or smaller company for a solution, or postpone the project until the market gets better. Tech decision makers like to be heroes, so cater to that.  Give them a solution that makes sense to them in a down market. A down market is a GREAT time for an emerging company to secure a beachhead against established players.

So how do you cater to them in a down market?  I suppose there isn’t one correct answer – it will vary depending upon your business, but … here are some thought starters based on what we’re seeing.

Startups can be more agile and creative with pricing and infrastructure. You don’t have 25,000 mouths to feed.  Yet … :) You have a handful.  Be aggressive with pricing – don’t try to get your whole nut on your first deal or two.  Get creative. Options are limitless – per seat, per transaction, per CPU hour, etc.  Are those up-front professional services fees getting in the way of closing the deal?  Waive them, and incorporate them into a transaction fee where the customer can pay for them over time.

Make your solution solve a real problem. In this market, the checks are being written to solution providers who can truly offer an efficiency or savings (of either time or money, or hopefully both).  If you aren’t doing this, you probably won’t last in the enterprise space. Don’t make your internal champion go back and explain why his or her boss needs to write a check to you.  Instead, arm them so they go back and show how much time and money they’ll save by bringing you in AND how painless it will be to get started. Everyone wants an on-demand solution these days – the days of NIH are shrinking.

If your solution doesn’t really solve a problem – make it solve one.

Get the deal DONE (especially if it involves a reference customer). If you can do this, others will dial down their perceived risk of entrusting a critical function to a startup provider.  It could even be worth losing money on a deal like that if you know it will open other doors for you – plus it slows your burn or at least helps you get to breakeven.

Put it in the cloud. Hardware is now a commodity.  It is a lot easier and cheaper to build a cloud solution these days.  Blade server prices are down to incredibly advantageous levels.  And if you can’t or don’t want to do it yourself, check out Scalr.net, which has a fantastic interface around Amazon’s EC2 service.

Enterprise services are the “ultimate mashup”. If you are an enterprise services startup, and you can effectively add value somewhere in a chain of web services, you have a decent shot at surviving this “Great Correction” as I’m calling the current market – but you are going to have to get deals done outside of the box.

Would love to hear some other thoughts …

Cheers.

Well, after four years of toiling, the StarPound project is finally seeing the light of day.  In the wee hours this morning, Wei Wang (CTO) and I published v1.1.0 RC1 of the open-source StarPound CORE platform to Sourceforge, and then published the new StarPound.net web site.  The team has been working nonstop over the past few months to make this launch date – we’re tired, but we made it.

You can read a lot more about the platform over on the site, but I’ll serve up a quick description of it here, and share a little bit about where we’re going.

StarPound CORE is a beautiful fusion of business process management (BPM), telephony (VoIP/SIP, PSTN/TDM, cellular, etc.), and web services (SOA).  What a mouthful.   In other words, the platform gives you an easy way to turn your organization’s business processes into web (or voice) services.  You can also use it to create any web service, not just those that are explicitly tied to some business process or telephony.

StarPound CORE is comprised of two key components: StarPound Studio and the StarPound Application Server.  StarPound Studio is a visual process modeling tool that is BPMN compliant (very cool), and built on top of the Eclipse IDE.  You create a visual model of what you want the voice/web service to do.  You can drag/drop really cool things like calls to external web services (e.g. Salesforce.com, Google, etc.), call control tasks, IVR tasks, etc.  Test it, then deploy it into the cloud on a StarPound App Server where it is ready to use.  The service can be initiated by a person, a phone call, an email, FAX, SMS, or web application request via SOAP or REST.

The platform is 100% Java/J2EE, but obviously you can invoke deployed StarPound services using whatever language you are using (via SOAP or REST calls).

There is a lot more to our vision than what I’ve described, but at its core (no pun intended), that’s what the platform does. You can read more about our vision here.

What’s even cooler than finally releasing the first release candidate of the platform is that we’ve also released two sample open-source applications that were built on top of the platform (StarPound PBX and StarPound Call Center).

StarPound PBX is a full-featured, free, open-source PBX that gives you pretty much all of the features you’d need out of a PBX for your office:

• Auto-attendant
• Voicemail (and web-based voicemail)
• Call menus
• Call hold
• Call forwarding
• Call routing
• Conference calling
• Hunt groups
• Unified messaging
• User directory
• Integrated voice response
• Call center support
• Web-based administration
• Open-source and free!

And, it is scalable because it is built on top of the StarPound platform.  Legally, I can’t mention any names, but one of the largest video game publishers in the world is now using StarPound PBX.  And why not? It kicks ass.

StarPound for Call Centers is also mega-cool. It provides mission critical functionality for call centers, like skills-based routing, automated call distribution (ACD), inbound/outbound, predictive dialing, workflow management, queue management, CTI integration, call recording/monitoring, screen pops, agent dashboard, supervisor dashboard, remote agent support, etc.  Again, free and open-source, and another really cool example of the types of apps you can build on top of StarPound CORE.

Right now we’ve got a ton of really exciting business and partnerships in the pipeline on the commercial side of things, and I’ll be writing some more about those things down the road as we work through them. But I can tell you that we are enjoying a tremendous push into the enterprise call center space, online marketplaces/exchanges, and scalable cloud services.

On a semi-related note, we’re also exploring opportunities to invest capital and resources in new startups that are in a position to take advantage of this new platform for creating disruptive applications within specific vertical markets.  More on this soon … but you can read the official particulars here.  We have a couple of deals that we’re already considering.

Now with all the cute cuddly launch comments out of the way, I’ll add this:  We have a long list of incumbent industry players that we’re going after, in multiple markets and sectors. And we’re coming fast and going for your jugular. Let the games begin.

Hats off to the whole StarPound dev team for this launch (Wei, Andrew, and the guys in St. Pete) – it has been a long time in the making, and I know that everyone is ecstatic right now about where we’re going with the company.  Good stuff …

Cheers.

Over the past 30 years, computers have changed dramatically in terms of processing power, graphics capability, and storage capacity. However, the one thing that hasn’t fundamentally changed is how we interact with the computer. We are still tethered via keyboards, mice, joysticks, and gamepads. Even the Nintendo Wii, as cool as it is, has you tethered to the experience through a wireless controller.

We’re doing our best to change that at PlayMotion, and in some cases, on a grand scale. Some of our experiences have hundreds, even thousands of simultaneous people collaborating together using natural gestures. We believe that the human body is the ultimate input device. Simply put, the human body is capable of performing movements and gestures that cannot be replicated by a traditional input device.

Gates sees diminished role for keyboards

PITTSBURGH – People will increasingly interact with computers using speech or touch screens rather than keyboards, Microsoft Corp. Chairman Bill Gates said.

“It’s one of the big bets we’re making,” he said during the final stop of a farewell tour before he withdraws from the company’s daily operations in July.

In five years, Microsoft expects more Internet searches to be done through speech than through typing on a keyboard, Gates told about 1,200 students and faculty members Thursday at Carnegie Mellon University.

Gates also said the software that is proliferating in various branches of science, including biology and astronomy must become even more advanced.

“They’re dealing with so much information that … the need for machine learning to figure out what’s going on with that data is absolutely essential,” he said.

Microsoft is trying to establish ties not only with university computer science departments but also with reseachers in other scientific areas “to help us understand where new inventions are necessary,” Gates said.

Gates plans to retire as Microsoft’s chief software architect in July and focus on philanthropy.

In my opinion, the next exciting point on the curve is the nexus of computer vision, gesture recognition and visual immersion. I’ll post some more thoughts on this soon …

Cheers.

If you will be attending the upcoming NASAGA 2007 conference, I’ll be delivering a cool keynote presentation (I hope!) on the trends surfacing with using interactive technology in the training and development space. The presentation will present my observations from the past (I spent quite a bit of time in the T&D/technology space with Cambridge Technology Partners and DCI), some observations about the present, and an overview of what I think the future holds (based upon current industry drivers). The latter two areas based upon our current work in the education market with clients at PlayMotion.

Link: North American Simulation and Gaming Association

I may turn the presentation into a video cast or something and post here – time permitting. Hope to see you there!

Cheers.

As pointed out by others, if this pace continues, it will easily be the best year for tech IPOs since 2000. Not bad.

I think another observation is warranted here. Most of the firms represented above are either in the network infrastructure or mobility sectors. These are both established markets that are resting firmly on solid ground. Contrast that to the IPO scene leading up to the shakeout in 2000, where you had a large number of companies going through an IPO that really should have never even been funded to begin with.

This is good.

Cheers.

For those who really still don’t get the movement, you owe it to yourself to watch this video. It is a bit long, but be patient – the payoff is worth it. This is done in the style of Lessig and Dick Hardt. Click the video clip below to play it.

Thanks to Michael Wesch, Assistant Professor of Cultural Anthropology at Kansas State University for putting this gem together.

Cheers.

Our informal discussions around scalability inspired me to jot down some of my thoughts on this issue, and how early-stage entrepreneurs can scale their technology platform from 5 users to millions.

Some Simple Rules

There rarely exists a set of rules that, if followed, will result in nirvana – scalability is no different. Every situation is different. However, these bullets summarize my general tenets, mantras, and beliefs for scaling a web-based application or system. The rest of this post will cover these in a bit more detail.

1. Abstract your logical architecture into at least 3 tiers (web, application, and data)
2. Even if you are only using one or two servers initially, think/design as if you have 50.
3. Designing with an SOA (services oriented architecture) in mind up front will aid you in scaling down the road.
4. Don’t overlook the scalability of your network infrastructure – your servers may be running at only 25% capacity, but if you don’t have the throughput and bandwidth to handle large amounts of traffic, those souped up servers won’t help you.
5. To help remove common/special cause variation – move system services such as DNS and e-mail off of production web/app servers and onto other, dedicated servers, and disable all non-essential services.
6. Scale vertically first, wherever possible, to control costs. But only do so if the benefits are greater than the cost. Early on, putting extra resources into an existing server is probably cheaper than procuring additional servers.
7. Scale horizontally when vertical scaling begins to produce diminishing returns
8. When upgrading/replacing existing servers, replace lesser horsepower ones first – and cannibalize components wherever possible (to control burn)
9. Performance tuning is not a one-time activity – it should be an iterative process
10. If you don’t have solid system administration capabilities in-house, outsource it immediately
11. Daily reporting is a must. You can’t know when/how to scale if you don’t know the landscape at any point in time.

What is scaling (or scalability)?
Simply put, scalability refers to an system’s ability to handle increasingly heavier loads from users (activity) without fundamentally breaking the way in which it operates. In other words, as you continue to add new users and expand your business, you want your application or service to be able to easily handle the increase without slowing down, or worse, breaking down completely.

An application (or platform) is considered scalable if it can continue to service additional users, through the deployment of supplemental hardware/software/resources, without seeing any significant performance hit from the user’s standpoint. Of course, very few systems in their early prototypical states will fit this definition. Your goal is to get the product to the point where it can be scaled in this manner, while reducing the number of potential bottlenecks.

Unfortunately, for business and IT managers, there is no single way to scale an application. Ah, if only there were actually a big red “easy” button. Each situation is different, given that there are so many factors that need to be taken into account. To make matters worse, sometimes, an application seems “infinitely scalable”, only to have a major bottleneck reveal itself down the road. This doesn’t mean the end of the world – it simply means that you have to adjust accordingly. The trick is, of course, to reduce the number of “midcourse corrections” that you will have to endure.

A Bit About n-Tier Architectures
Before we dive in too deep, I should probably throw out a note or two on n-Tier architectures. If you are an IT weenie, and understand this concept, skip to the next section. Otherwise, hang with me.

In the old days, applications were deployed onto servers, and when a bottleneck was encountered, the physical resources in the machines were expanded. This was the extent of scalability. Then, some bright engineer realized that if you split a system into two “tiers”, you could distribute the workload a bit. Voila – the birth of the client/server revolution. Eventually, though, systems began to grow so large that they needed something else in order to break through the inherent bottlenecks of a 2-tier system. An even brighter engineer realized that there was no reason to stop at “2″ tiers. You could have an arbitrary number of tiers in your system. Thus, the birth of the “n-Tier” architecture (“n” representing some arbitrary number of tiers).

An “n-tier” application architecture is characterized by the functional decomposition of applications, service components, and their distributed deployment. By breaking a system down in such a manner, it provides for improved scalability, availability, manageability, and good resource utilization. A “tier” itself is nothing more than a functionally separated hardware and software component that performs a specific function. Whew – what a mouthful.

Typical “tiers” include:

• Web server tier: provides HTTP protocol support (i.e. handles web requests)
• Application server tier: provides support for web services, business logic, etc. (e.g. web services/SOA)
• Database tier: provides data storage and retrieval support (e.g. Oracle, MySQL, mainframe/VSAM, etc.)

Note: don’t confuse these “tiers” with application “layers” (presentation layer, data layer, business logic layer, etc.) Tiers are architectural in nature, whereas “layers” are generally code/library specific.

The important thing to know here is that in an “n-tier” model, a system has been broken up into various levels of functionality, each capable of some degree of horizontal scaling. Which brings me to my next point …

Horizontal vs. Vertical Scaling
When you hear people talk about “scaling horizontally” they are essentially referring to the ability to add new servers to a tier to allow it continue to provide uninterrupted service in the face of continuously increasing usage. For example, your database is chugging hard and heavy, so you can add new servers to the database tier to distribute the workload. If your web server is bogged down, you can add new web servers to do the same. This also affords you with a nice layer of failover as well. If one server experiences an issue (even to the point where it crashes), you have other servers online in that particular tier that can still provide service. This concept is becoming increasingly important as more and more systems are being deployed using SOA models (service oriented architectures).

One great example of horizontal scaling can be seen in Amazon’s new “Elastic Computing Cloud” (my old friend Chris Brown is a patent co-inventor and senior engineer on that project).

Vertical scaling, on the other hand, is where you extend/expand the physical resources in a server itself. For instance, your database server is getting way overworked, hitting the swap space early and often. You can “vertically scale” that server by simply adding more RAM, faster hard disks, better CPUs, etc.

There are benefits/pros/cons to each type of scaling. Obviously there is a cost associated with both. Horizontal scaling is theoretically infinite, whereas vertical scaling has an obvious ceiling (there is only so much horsepower that you can derive from a single server).

Horizontal scaling only makes sense if the service you are attempting to scale was designed to be extended in this manner. For many third-party applications, such as a database server, this will be the case. Of course, if you are designing the software, you will want to take this into account as you build it.

Think of hardware as simply a vehicle (perhaps a bus) for your software, your real service. If the bus gets too crowded, you add another bus to the fleet. However, not all buses will go to the same destination, so those buses need to connect together in order to get information from point A to point B within your architecture. Voila, you have the meager beginnings of a service-oriented-architecture (SOA).

A Scalability Example
Let’s set the stage with a typical early-stage example, and we’ll try to scale this application theoretically as the business scales.

John is an aspiring technology entrepreneur who has developed a really great online service called WidgetFire. WidgetFire is brand new, so there aren’t many users yet. John has built this product in his spare time, and is bootstrapping the business via his day job as a software engineer for another company. To keep his costs down, he has a single web server (built from extra parts), and he has it co-located at a local data center with a basic level of service (1U single rack space, 1Mpbs throughput, unlimited bandwidth, for probably < $100/month). On this single server, he is running Apache and MySQL together, along with the normal services (bind/DNS, sendmail, etc.)

So far, scalability is not a concern to John. But that is about to change. In a big way.

Over a period of a few months, John leverages word-of-mouth marketing and manages to aggregate 25,000 registered users for his service. The server he is running (which he’s affectionately nicknamed “Seabiscuit”) is holding up fine, but is beginning to feel the strains of all of those new sessions and database queries. Additionally, his automated e-mail notification list is starting to add to the system load, as now the server is sending out thousands of emails a day.

John “vertically scales” his server by adding some additional RAM and performing some additional performance tuning to the database. This buys him time. But not much.

Then, WidgetFire gets a mention in a prominent tech blog, and the next thing John knows, he has 100,000 registered users for his service. His server is on its knees, and practically unresponsive. Sadly, the data center staff isn’t much help – after all, he is running under a pretty basic co-located hosting plan – and they have bigger problems to deal with than the occasionally unresponsive Seabiscuit.

So John then moves to a very rudimentary n-Tier architecture. He moves his MySQL database over to a separate server, which frees up resources on the old web server. Now, the system is humming along smoothly. But after a few months, WidgetFire is quite the rage on college campuses, and John must once again address how he is going to facilitate additional traffic.

He does some vertical scaling on the database server (adds new RAM, faster disks, etc.), but it isn’t enough. The new user signups are coming too fast and furious for his 2 server setup to handle.

This particular nexus is where many startups begin to experience some tough scalability issues. The “easy” scaling options have already been exhausted (vertical scaling on one server, splitting the database off into a separate server.)

Fortunately for John, his traction has caught the interest of a handful of investors, and he is able to secure a small round of outside capital.

To get to the next level, John implements a load balancing router, an extra web server, and an extra database server. He configures the load balancer to distribute incoming web requests evenly between his two web servers. Additionally, he configures the original database server to be a “master”, and the new database server to be a “slave” server. While all database write operations occur on the master server, John realizes that most web requests that require database access will be for “reads”, so the slave can offload some of that workload from the master. Voila – he has scaled even further!

A few more months go by, and John makes the cover of Wired Magazine. VCs are clamoring to pour their cash into WidgetFire. While John basks in this glory, he doesn’t realize that his little server farm is becoming overwhelmed by the sheer success of his venture. To make matters worse, while John is putting together a plan to scale even further, his slave database server crashes, leaving only the one original database server in operation. WidgetFire is basically dead in the water.

John brings the slave server back online, but he realizes that more must be done. He adds an extra web server to the web server tier, and adds an additional slave database server to the database tier. But he doesn’t stop there. John realizes that his actual application, which is comprised of a mish-mash of Java, PHP, and Perl, is utilizing the vast majority of CPU time on the web servers themselves. John decides to move from a 2-tier model to a 3-tier model by implementing an “application services” tier. He moves this code off of the web servers and onto several new servers.

John realizes pretty quickly that his original code design really wasn’t architected for this type of model. He has to spend a couple of months retrofitting his old code to fit into more of a “web services” model. He is now surrounded by burgeoning IT costs, hosting fees, and system complexity. All of a sudden, his “Google-ready” venture is giving him a headache, and isn’t much fun anymore.

Of course, had John anticipated the steepness of his growth curve ahead of time, he could have designed his system with it in mind, and avoided at least some of the headaches.

Can’t money be thrown at the problem?
Sure. To a point. All things cost money, of course, whether it be labor (people), hardware, or software. Most systems can initially be scaled to a sufficient level by simply adding more hardware, or expanding the resources within the server. But that only gets you so far in most cases – at some point the logical architecture of the system needs to have been designed with scalability in mind. If the architecture isn’t scalable, you are either going to hit a ceiling, or spend way too much money to scale it (and even then, there are no guarantees).

It isn’t a question of whether or not money can be thrown at the problem – it all costs money. The question is how much money are you going to have to spend to scale it? Obviously, there rarely exists an endless supply of capital that can be leveraged to solve a scalability problem – especially in the startup realm. Clearly, you want to be able to control your tech spend (which is a big part of your overall burn).

The bottom line is this: If you plan for scaling initially, instead of throwing a bunch of crap together and hoping it will get you to point “B”, the less money you are going to spend as you scale.

To properly address scalability, you have to take a holistic approach, and examine your architecture, your software components, and your hardware configurations. Then you have to deploy capital in an intelligent manner. Otherwise, you could end up like John in our fictitious example – sitting on a pile of code that really wasn’t designed to split up into services across an n-tier architecture. And that, my friends, represents a serious misuse of capital.

If you are scaling by adding new hardware – that is generally a good problem to have – that hopefully means your business is expanding. However, if you are having to rewrite large amounts of code in order to scale – you’ve likely made some very serious mistakes. Too many of those mistakes, and you’ll be dead in the water. You see the latter quite often in the startup world, again, as there is so much pressure to slap something together and get it out the door.

Architectural Concerns
It all starts with the blueprint of your architecture. Note, I am not referring to your functionality matrix/map, your application requirements definition, etc. I am talking about your physical and logical architectures.

When I say physical architecture, I am referring to the physical hardware components that make up your network, application, etc. Things to consider:

• What are your network capabilities? Throughput, bandwidth, etc. Are these scalable from a cost and availability standpoint?
• What are the capabilities and specifications of your web servers, application servers, database servers, etc.?
• How many of these servers do you have at your disposal? Are they dedicated or shared?
• What are your capabilities for handling increased DNS requests, e-mail volume (in and out), etc.?

When I refer to your logical architecture, I am referring to the way in which your various software components connect with and layer into one another. Things to consider:

• Are you properly segregating functional aspects of your application into easily managed and scalable services (e.g. SOA)?
• Are you isolating those services in unique hardware or operating environments?
• Are you building for the short-term, but planning for the long-term (i.e. how extensible will this thing be down the road?)

If you are in startup mode it is very easy to fall into the trap of “getting started” – pushing code and charging up the hill. “Get-to-market” pressure from investors rarely helps. However, many such efforts are met with stiff resistance once the entrepreneur realizes that the “hill” he or she just conquered is actually but a small plateau on a mountain comprised of increasingly steeper slopes. It all starts with a good plan in place!

Having the right architecture in place doesn’t guarantee that you won’t have to eventually add more hardware or write more code. Actually, in the early stages, you may actually spend more money (especially if you are deploying web services on their own servers, etc.) However, the proper architecture allows you to get the most out of out that new hardware and software, as you will be plugging it into a framework that was built with it in mind.

A Note on System Services
Before you start diving off into creating a true services oriented architecture, do yourself a favor and split your system services off accordingly. Things like DNS services and e-mail should be move off of and away from your application’s production environment. I bring this up because more often than not, you find system services being co-located on production web servers.

It is hard to get a sense of an application’s true load on a physical machine when you have a million other things running on it. This becomes even more important if you are planning on using tools like Six Sigma to measure scaling and availability metrics, as you will need to do everything possible to remove potential causes of common/special cause variation.

The Importance of Performance Tuning
Another very important aspect of scaling is performance tuning. Performance tuning is essentially the art of tweaking and fine-tuning your various applications and services in order to maximize their operational efficiency. There are more ways to performance tune a system than you can possibly imagine. Some obvious examples would include:

• Tuning your database server to use more or less RAM (for buffering, sorts, key lookups, etc.), adding indexes on data tables to decrease query times, etc.
• Optimizing your web server’s socket configuration to improve connection times

Some not-so-obvious examples might include:

• Enabling DMA (direct memory access) for a server’s IDE drives to improve seek times (you should be using 15K RPM SCSI drives, but if you aren’t … )
• Making sure your OS kernel (for UNIX/Linux) was compiled with only the services and features you require (i.e. remove kernel bloat)
• Mounting filesystems with the “+noatime” flag, which will prevent the OS from updating the (mostly) meaningless “last access timestamp” of every file you read or write
• Partitioning your OS so that system files, logs, etc. are on their own partition (or even a separate IDE channel or SCSI bus)
• Using the “strip” command to remove debugging/profiling data from common executables to lower their memory footprint (On UNIX systems)

I have known people who have spent thousands on new hardware, only to realize later that their applications or server/OS were simply not optimized. if you don’t have the requisite skills in house to do performance tuning, then outsource this function immediately – it is that important.

Finally, I should mention that performance tuning should not be viewed as a one-time activity. You should routinely profile and tune your systems (both hardware and software).

Network Scalability
Another thing to stay on top of is your service level agreements with your data center or hosting provider. There are fundamentally three or four things to keep in mind:

First, if you are bootstrapping a startup, and you are using a low-cost, shared server, you need to move to a dedicated server solution as soon as possible. Trust me.

Next, make sure that you have the ability to quickly secure additional rackspace and bandwidth as you need it. Having your data center tell you that you are going to have to suffer major downtime because they have to move your servers to a “bigger rack” is probably not a good thing.

Third, make sure that the connections from your servers to the net are burstable. When you have those huge traffic spikes because someone put a mention of your site on digg.com, Slashdot, etc., you’ll want to be able to handle the temporary increase in traffic (without necessarily incurring a large bandwidth bill).

Finally, make sure that your throughput is not capped, or if it is capped, make sure it is capped higher than you think you’ll need. Don’t confuse bandwidth with throughput. Bandwidth refers to how much data your connection can transfer over a period of time (e.g. 100 gigabytes per month, etc.) Throughput, on the other hand, refers to how much data can be flowing through your connection at any given time (e.g. 10Mbps). Think of throughput as being the thickness of your server’s pipe – obviously, a lot more can flow through a garden hose than a soda straw. Same analogy.

To illustrate the point on throughput; a few years back we had a system that began to be wickedly unresponsive. The CPU loads on the server were only about 35-50%, so it didn’t make a lot of sense. It turns out that the connection had been capped at 10Mbps. Anything over 10Mpbs at any point in time had to basically wait in the queue. This caused perceived “slowdowns” by users. Raising the throughput cap remedied the problem, of course.

On a final note, I want to voice my support for co-locating servers rather than using a full-service hosting provider. If you have the skillset in house to maintain the boxes, using co-location can save you some money, and give you more flexibility. Again, in startup mode, every penny counts. If you are in or near Atlanta, I highly recommend my friends down at Capital Internet. I’ve co-located and hosted servers with them for 6 years now, and it has been a very enjoyable, hassle-free partnership.

Advanced Topics
There are a lot of advanced topics that come up in discussions about scaling applications. Things like caching, high availability clustering, network storage (via SANs), and distributed networks. Obviously, those are very specific areas that are beyond the scope of this already ridiculously long blog post. Suffice it to say that there are some very advanced (and expensive) toys out there that can make scaling a lot easier. However, the vast majority of applications/services can be scaled to rather massive proportions if you simply follow the bullet points at the top of this post.