Thoughts on Twitter #9: Twitter is getting too big to fail, and that’s a whale of a problem for all of us

thotsontwitter.jpg

[Part of the RexBlog "Thoughts on Twitter" series.]

Anyone who knows me “online” knows this: I’m a big fan of Twitter. At Hammock, the service has long-been a part of a standard set of content distribution, aggregation and conversational tools that play a part in nearly every content marketing project we develop or manage.

I personally use Twitter throughout the day and evening to point to fascinating, helpful or ridiculous things I run across from the news feeds I use to track the topics I follow for work or pleasure. Twitter is a place I use to attempt to balance a little humor to the more serious work I may be engaged in throughout the day.

Twitter, like the internet itself, has morphed from novelty toy to essential tool in just a few years. While the world was still giggling at its name, Twitter became integrated into the information infrastructure and flow of such activities as emergency and crisis response, running a business and investing. In other words, activities that are not just “fun,” but are “mission critical.”

twitter bird failing

Ironically, it’s what I’ve jokingly called, “the inability of anyone to understand Twitter” that has enabled the service to become so intertwined in so many important aspects of our lives and work. Twitter is so flexible and accessible and freely available and open for third-party development, that It has become, metaphorically speaking, an operating system for an entire form of media that we did not realize was so important until it appeared in the form of Twitter: A drop-dead simple means of distributing “from one person to many people” short messages that can be sent and received using any computer or mobile device.

Twitter has become the electricity powering entirely new forms of engines of communication, conversation, transaction and collaboration.

All of that is great. What I’m beginning to fear, however, is that Twitter, too, has also become the electricity grid through which all of this power must pass.

In other words, I believe Twitter (the service, not the company) is quickly assuming a role in our lives and work that is making it “too big to fail.” I am also moving to a belief that too many people, organizations and transactions depend on “the service” Twitter for this new form of communication — and that makes the network through which this communication must pass too important to be controlled by one company. Or, to put it another way, one company should not bear the responsibility for all that is being done via Twitter.

I don’t have this opinion because of any desire to see the demise of Twitter, the service or the company. I’m a capitalist and I would never advocate any sort of “socializing” of the Twitter service — or something like a forced “opening” of the Twitter service in a way that would provide the distribution of “tweets” without them ever hitting any Twitter servers.

That said, I feel sure, even if the underlying technology enabling the distribution of messages were “open-standards” the fortunes of the company, Twitter, would still flourish as it would be the primary beneficiary of having created the infrastructure. The company, too, would have an asset worth billions of dollars because of the number of “subscribers” who would choose to continue to have access to “the network” via Twitter. In other words, if Twitter, the company, only provided one of several means of access to Twitter, the service, then I believe it would still be a tremendously valuable and important company.

But I’m not ready to suggest that the “access” part of Twitter be separated from the “network” part of Twitter. Not yet. There is plenty of time to avoid that.

And I have to admit: I’m even shocked at these thoughts coming from me. Typically, I’d be in the gung-ho, “let the marketplace decide” camp when it comes to things like this. However, as I wrote on this blog in March, 2009, there are certain things I no longer believe in, the first being, “Anything too big to fail.” I said then:

“The whole ‘bigger is always better’ thing has now been exposed as a nice theory, but a failed reality. Why? Because all the algorithms and information technology and most brilliant programming in the world can’t overcome the bugs of greed, hubris and randomness that can best be summed up in the vernacular, ‘sh*t happens.'”

Arguably, Twitter is still a “little” company with very little revenues, so how could I be so ridiculous to suggest it is to big to fail? Well, I can’t. Now.

But I remember when e-mail was about where Twitter is. Back then, if you and I wanted to exchange e-mail, we both had to subscribe to the same service, something like CompuServe. But then, something bigger came along — an open network called the Internet, and the rest is history (including spam).

Today, despite spam, email is probably the most used, and ubiquitous one-to-one communications channel in our business and personal lives. And because the network is not limited to one provider, our email can move around the globe via an endless number of paths.

For the post email type of service that enables one-to-many short messages via computer or mobile device, Twitter is the only player who matters. And each day, as more and more new and existing services integrate Twitter messaging into their products, Twitter becomes even more entrenched into the molecular level of a unique form of communication that is so flexible, everyone who uses it can interpret it differently.

But those who have adopted it to help run their businesses or respond to emergencies have, from time to time, a wake up call about the downside of having just one dominant provider of such a mission-critical service.

Last week, the users of Twitter experienced what the company officially called, “incidences of poor site performance,” a euphemism for “Twitter didn’t work” — a seemingly familiar situation Twitter users have nicknamed “fail-whaling” after the whale and bird illustration that appears on the “over-capacity” page.

In reality, Twitter probably has a stellar “up-time” performance history compared to other major internet services. Indeed, its uptime stats demonstrate that, over the past year, the service is online an average of 99.74% of the time.

Yet, as active Twitter users know, the downtimes seem longer than a matter of an hour or two each month as they tend to hit during times of usage spikes — the very times when its importance is measured most.

1. It is time for people who are outside the tech-bubble (where such issues are actually debated already) to begin considering the need for redundancy or alternative solutions to the service provided by Twitter.

I have no desire to stop using Twitter. Indeed, I doubt I’d stop using Twitter, if given the option. However, the function it serves at critical times needs to start receiving the consideration we give to all voice and data tools and channels we use to run our businesses and protect our communities.

2. Twitter needs viable direct competition.

The marketplace — including consumers and users and even the company called Twitter — has a vested interest in having alternatives to Twitter. Competition and choice are good. And we (including the company, Twitter) should prefer to see competition or open-source or standards-based approaches challenge Twitter than to see the inevitable calls for regulations (or worse) when the “sh*t happens” that will cause Twitter to fail at a time of national or international emergency.

We need to learn about and support any open-source alternatives to Twitter like StatusNet, the platform that powers the service identi.ca and other similar services.

Or, perhaps there needs to be an open standard whereby a more explicit protocol for “twitter-like messaging” is agreed-upon that will be a part of the Internet Protocol Suite. (I apologize to the geeks who are going to read that sentence and feel the need to lecture me on the history, role and future of every protocol related to anything Twitter does — or that something already exists. Such may exist in theory or concept, but I’m actually talking here about (okay, let me put this in a metaphor) the way we can read because the electric lightbulb exists, not the “current wars” fought over AC and DC. More importantly, you’ve thought about this stuff. The people who read this blog haven’t.)

Some could, in response to many of the issues I’ve raised in this post, argue convincingly that other standards or protocols or formats like RSS, or service features like Facebook status updates or Google Buzz, or software platforms like Tumblr or Posterous are viable alternatives to Twitter. They may be, sorta. But for the core thing that Twitter does — what I’m talking about here — they aren’t.

Bottomline: Just like everything with Twitter, I fear we won’t know the consequences of placing all our eggs in the Twitter basket will be until they are crushed by the Moby Dick of fail whales.

  • http://gravitationalpull.net/wp/ ampressman

    Agreed but unsure how to proceed. Twitter seems to have vaporized my first 2,000 or so tweets yesterday with no warnings or explanation.

  • http://icosidodecahedron.com Michael Kozakewich

    Fine, I'll add a comment, too.

    The biggest problem I can see is that we would need to access hundreds of servers, and have possibly thousands of servers accessing ours, in order to get our tweets. Like RSS, we'll have to have buttons with all sorts of addresses, and will have to add those to our tweet application. If someone changes the location of their stream, we may have to manually change that location in our tweeting program.

    With something as massive as Twitter, we do need at least a semi-centralized source. This way, we get all our tweets with one or two server connections.

  • jimvoorhies

    Well, I just learned that you should never click the little down arrow nest to the Disqus logo if you're in the middle of a post. It deletes everything. I now officially hate Disqus as a bad user experience.

    However, I see the value and immediacy and flow of information from Twitter as a necessary connection professionally. LinkedIn tries to do that, but it is passive. It is possible to communicate there and discuss there, but it is vestigial bulletin board communication (ooh, I like that phrase. :) ) and a not-distant-enough relation to 300 and 1200 baud modem BBS life of the 1980s.

    Twitter is active and (damn it) constant. I follow an odd mix (Twitter is personally reflective, I fear.) of locals and people in the user experience field and each day is a constant flow of informative links to articles, opinions and leadership thought in my field. But there is too much flow of information. If I go through the night and not check twitter when I have to get up to pee in the middle of the night, catching up is possible the next morning, but only barely. I'm worried about missing something important almost as much as I worry about not sleeping well. It's probably not a valid fear – I still continue to learn even though I'm not up on 24 hours of Tweets, but is that enough these days?

    But I'm unsure how we can create a competitor without doing ourselves damage? Will there then be Twitter and Flitter and Flubber, all of which will have UX or UI participants I must then follow and – worse yet – coordinate together into a unified field of knowledge? Or do we just need to take the Ashton Kuchners of tweetland and stick them on a special bank of servers to reduce the loads on our Twitter – the real Twitter? Perhaps there is a second or third option, yet undiscovered to solve this Too Big problem, because it is valid. Perhaps tweets can be simply sent to central Twitter and RSS feeds used provide us with the feed of tweets we want to follow. Does changing how the distribution occurs give freedom to the fail whale?

    On the other hand, since you've now convinced me of thinking of Twitter as something that's too big to fail, like AIG or GM, I'm considering starting a hedge fund based on Kuchner's and Oprah's tweets, combined with some real losers' tweets – collateralized tweet swaps. I ought to be a millionaire in no time.

  • http://rexblog.com Rex Hammock

    I hope you saw that WSJ op-ed piece recently that Scott Adams (Dilbert creator) wrote about his investment strategy: He only invests in companies he hates.

    Without getting too wonkish, I'll simply say, the ways that e-mail and instant-messaging are no longer proprietary products “owned” by one company provide a lesson in how “competition” to Twitter may not be a company.

  • http://icosidodecahedron.com Michael Kozakewich

    Fine, I'll add a comment, too.

    The biggest problem I can see is that we would need to access hundreds of servers, and have possibly thousands of servers accessing ours, in order to get our tweets. Like RSS, we'll have to have buttons with all sorts of addresses, and will have to add those to our tweet application. If someone changes the location of their stream, we may have to manually change that location in our tweeting program.

    With something as massive as Twitter, we do need at least a semi-centralized source. This way, we get all our tweets with one or two server connections.

  • jimvoorhies

    Well, I just learned that you should never click the little down arrow nest to the Disqus logo if you're in the middle of a post. It deletes everything. I now officially hate Disqus as a bad user experience.

    However, I see the value and immediacy and flow of information from Twitter as a necessary connection professionally. LinkedIn tries to do that, but it is passive. It is possible to communicate there and discuss there, but it is vestigial bulletin board communication (ooh, I like that phrase. :) ) and a not-distant-enough relation to 300 and 1200 baud modem BBS life of the 1980s.

    Twitter is active and (damn it) constant. I follow an odd mix (Twitter is personally reflective, I fear.) of locals and people in the user experience field and each day is a constant flow of informative links to articles, opinions and leadership thought in my field. But there is too much flow of information. If I go through the night and not check twitter when I have to get up to pee in the middle of the night, catching up is possible the next morning, but only barely. I'm worried about missing something important almost as much as I worry about not sleeping well. It's probably not a valid fear – I still continue to learn even though I'm not up on 24 hours of Tweets, but is that enough these days?

    But I'm unsure how we can create a competitor without doing ourselves damage? Will there then be Twitter and Flitter and Flubber, all of which will have UX or UI participants I must then follow and – worse yet – coordinate together into a unified field of knowledge? Or do we just need to take the Ashton Kuchners of tweetland and stick them on a special bank of servers to reduce the loads on our Twitter – the real Twitter? Perhaps there is a second or third option, yet undiscovered to solve this Too Big problem, because it is valid. Perhaps tweets can be simply sent to central Twitter and RSS feeds used provide us with the feed of tweets we want to follow. Does changing how the distribution occurs give freedom to the fail whale?

    On the other hand, since you've now convinced me of thinking of Twitter as something that's too big to fail, like AIG or GM, I'm considering starting a hedge fund based on Kuchner's and Oprah's tweets, combined with some real losers' tweets – collateralized tweet swaps. I ought to be a millionaire in no time.

  • http://rexblog.com Rex Hammock

    I hope you saw that WSJ op-ed piece recently that Scott Adams (Dilbert creator) wrote about his investment strategy: He only invests in companies he hates.

    Without getting too wonkish, I'll simply say, the ways that e-mail and instant-messaging are no longer proprietary products “owned” by one company provide a lesson in how “competition” to Twitter may not be a company.