Is there a tipping point when rewriting code should be favored over refactoring ? It is often hard to decide. There are some important factors that can help. Code smells cluster, whether as bugs that are detected, or potential failures lurking in immature code that may break during future updates or maintenance. Steve McConnell wrote (in “Rapid Development”) that “Error prone modules tend to be more complex than other modules in the system, less structured and unusually large. They often were developed under excessive schedule pressure and were not fully tested”. He also quotes indicative studies that these modules are much more expensive to develop. Steve’s metric is 10 defect per 1000 lines of code to mark a module for further work. If it appears to be an error prone module, he advises rewriting it. Other indicators are large concentrations of bugs, large number of checkins, or large amounts of fix time spent in comparison to similar modules. While error-prone modules are typical in traditional development, agile practices like well executed TDD should minimize them. High cyclomatic complexity of code is also a good indicator of the need for simplification.
The hard thing is determining the tipping point between refactor and rebuild. The easy thing is that because of clustering, there is often a lot of low hanging fruit, and focused effort on a small portion of the code should have a massive impact on the code quality.
For more on bug clusters and other productivity ideas, see my Google tech talk “Building Software Smarter”, http://www.tinyurl.com/80-20-rules