Firefox is becoming faster and faster. The most recent versions have made significant improvements, both in memory usage and in responsiveness. It can get better, obviously, and the team is working on it.
But there is something you can do right now that has the potential of significantly speeding up your browser, depending on how many tabs you have open and the websites you use.
Wladimir Palant, of AdBlock Plus fame has created the "Suspend background tabs" add-on
for Firefox. The add-on kills scripts from background tabs stopping them from eating up CPU time.
The add-on doesn't affect long background downloads and neither Flash content. With the add-on installed, only the visible tabs or, optionally, pinned app tabs will be allowed to run.
Note that the add-on is still somewhat experimental, but users have been reporting good results so far.