I'd actually turn the title of this talk around: it's never too early to fight your legacy. Máté Nádasdi from Ustream did a thorough walkthrough of the process of cleaning up their frontend mess and creating a nice, layered, modularized, testable codebase that's ready for continuous integration.

Lots of examples, hands-on experience from a site serving millions of users, tried out tools and valuable advice. Worth to watch if you maintain a javascript codebase that's starting to age - and show me one javascript codebase that doesn't (except for a few months from launch maybe).

Update: the recording of this talk is available here:

Video streaming by Ustream