In Lori MacVittie's latest blog, "The Devil Is In The Details" she not only bestows upon me the honor of my own Theorem (yeah, in your face Pythagorus) she also gives a number of great examples of some of the necessary dimensions of scale beyond just the number of servers.
But besides things like networking and bandwidth, there is a softer side of scale that is equally important: people. You need a certain critical mass of support, billing, operations, development teams, security, sales, developer support, evangelists, etc. to create a viable service offering and economies of scale apply to these dimensions just as in hardware.
There may be niche markets where small providers can provide some unique value-add (specialized security procedures, vertical focus, non-standard technology stacks, etc.) but in general I think the dominance of scale is inevitable. As a developer I love the flexibility and programmability of cloud computing services but ultimately the trump card for businesses is cost and the best way to drive cost down is via scale.
Over the next five years, I think the majority of cloud computing will happen on public clouds and that the public cloud landscape will consist mainly of a relatively small number of big players who will be able to scale their services, both the hard side and the soft side, to achieve the economies of scale required in the marketplace.