Start with small experiments that solve real problems that people are having. See what works and build on that.
This is the opposite of "Get Big or Get Out." One of the benefits of the web is that it's possible to make smaller companies that solve specific problems without costing a fortune in startup costs. Especially with technologies like Google App Engine and the like, it's becoming easier and easier to create full-powered, scaleable systems from dorm rooms and garages.
The companies that win will be the most fun places to work.
Contrary to popular belief, technology is neither the problem nor the solution. Technology is never a measure of itself and always a measure of us. (This has always been the best evaluator for science-fiction stories).
Leaders tend to form shells around themselves, for protection from being overwhelmed. But then the leader cannot grow outside the border of the shell. And, cut off from new energy, what's inside begins to shrivel and die.
Your strengths, the things that make your business successful, are the very things that will become weaknesses and bring you down.
I think the actual saying is "go big or go home". The idea is that if you are going to try to do something you should commit and make a huge investment in order to attain a large market share. Start small is the opposite because you make a smaller investment and use returns to build your stake.
I think the former comes from the past of manufacturing. For example, the start up costs for metal stamping of cars is very high. Starting small isn't really an option because you cannot recoup and a year later you will be worse off even if you sell every car. This is probably still true for things like large LCD screens. You won't see many small quantity producers of such things.