The latest example of the advancements in malware was the Flame cyber-espionage app discovered last month. The creators obtained a digital certificate that allowed them to sign their code as coming from Microsoft to evade detection in their attacks on Middle Eastern governments.
Antvirus systems alone can catch known malware, but new apps go undetected because they do not contain the watched-for code sequences.
Hackers have multiple evasive techniques against AV technology. Those include encrypting the malicious file or compressing it, so it has to be unpacked before it can be checked, an additional step not normally performed by AV software.