The most dangerous factor for productivity is the void space between your iterations, the time spent waiting for assets or code to build. Fortunately there are only really three ways to measure it: instantaneous, one minute, or coffee break.
Instantaneous results are key to keeping yourself in the zone. You stay focused and engaged, the possible changes you want to make never have to leave your head because they'll be soon tried.
If you're in the one minute range you're nearing the danger zone but your concentration is still salvageable. If you're waiting for a minute between builds (and redeployment) you might check your email, visit Reddit, read another bit of that AltDevBlogADay article you were working your way through on your last build, etc., but the problem hasn't entirely vacated your brain.
Anything past a minute is disastrous to productivity. If you have to stop work because of a five/ten/fifteen minute build time then you're going to find something else to do. You'll go for coffee. You'll start working on another project. You'll finish writing your AltDevBlogADay article, and maybe the one after that. We all know programmers are people who turn coffee into code, but if they have too much coffee they spend all their time in the washroom.
Like a lobster in a pot of warming water your build times may increase (and your productivity may plummet) without anyone noticing. Before you know it you've gone from sculpting code like it was fresh, wet clay to being in the break room, adjusting the coarseness of the grind used in your French press for the best cup of rocket fuel.
Proponents of dynamic languages are all over this, you don't have to sit in front of a Python or Lisp REPL for very long before you realize how fantastic it is to have your results before your brain has even processed the "lift-pinky-finger-off-of-enter-key" message.
Much of game development is stuck with the compile-link-redeploy, as cool as it would be to attach SLIME to a running Lisp image on your PS3, it might not be practical for your current project.
Work costs time. To consume less time you must do less work.
Most build optimizations revolve around compilers being pretty stupid. If your source file tells the compiler to include Windows.h it won't check to make sure if it's needed, it'll just go and paste in its contents right where you told it to. And after it's finished the inclusion it has to go and parse the resulting 3mb of code.
A precompiled header is one way around this. Put all your includes into the PCH, have the compiler process it into whatever convenient intermediate form it so chooses, and then refer to it, saving the cost of repeatedly including some potentially large files.
If precompiled headers aren't an option because they just don't work for your code, or you're using CodeWarrior, or whatever, there's bulk/master/unity files which involve you #include'ing all your source files into one file, and compiling that one file. That said I really don't like bulk build files as they diminish three of the five meanings of the static keyword and also because they can lead to globbing together data that might otherwise be considered dead by the linker.
Because I develop middleware I don't have the luxury of insisting my users use precompiled headers or bulk builds. They almost always do but I still have that restriction in place. So when it comes to optimizing my builds I do it the same way I would when optimizing code.