Troubleshooting VoIP issues over ASA/PIX/FWSM appliances.

Recently I've been again put in a situation where I had to troubleshoot voice over IP issues via firewall appliances by Cisco.

Here's a brief summary of types of issues:
- No Voice (including one way voice)
- Delay
- No communication
- Features not working (MoH etc).

Now I will be the first to admit - I am voice-over-IP-agnostic, I've touched CCM or CME once or twice in my life. I'm aware that h323, sip and skinny exist.

From my perspective only two things exist in voice over IP.
- signaling
- actual payload (RTP).

So what you should know about how ASA/FWSM interacts with voice traffic.
0. Before going into voice troubleshooting check routing. In many cases a few pings can save a few hours of "missing voice" or "broken signaling"
1. First of all you most likely want to inspect your signaling traffic to/from the internet
a) Be aware that some implementations do not use default ports:
Prime example - SIP - default udp/5060 but known also to use tcp/5060 and udp/5061 (yes for signaling)
b) Be aware that if you're running voice traffic over VPN tunnel you most likely do not need to inspect traffic (everything normally should be permitted). Inspecting over VPN is waste of time (delay) and can account for some one way voice issues.
2. What does the actual inspection do:
a) Dynamically opens pinholes in your security policy to allow incoming traffic (or outgoing if relevant)
b) Pre-allocates connections and translations for traffic.
Thing to take into account here
!!!!!!
If a given IP/phone is not mentioned in signaling the traffic will NOT be allowed in. This comes over and over with music on hold from secondary CCMs. Since none of the parties actually mentions that this particular CCM will be sending MoH the traffic is neatly being dropped because of security policy.
!!!!!!!


3. ASA/FWSM inspects AND changes payload of signaling packets - and that's a good thing.
Firewalls will change internal IP address to public ones according to translation rules in place.
Be mindful when changing inspection or translation rules. "clear xlate ..." or "clear conn ..." has a big chance of recovering if something went wrong after reconfiguration either in inspection rules or NAT.

4. RSVP packets. They usually contain IP options. ASA 8.0 will drop them. Check out ip-option inspection in ASA 8.2.

5. Signaling works but voice does not go properly (dead silence and one way voice issues).
This is most likely caused by a problem with inspection or some additional policies.
Ie. Firewall should automatically open pin holes in security policy for both way RTP traffic.

Check
------
show conn detail add x.x.x.x
show x det | i x.x.x.x
------
Where x.x.x.x is the source or destination of either signaling or voice.

You should have also enabled logging on informational level and monitor logs for entries mentioning IP address of either source of destination. 

Check also those (pick ones interesting)
h323
--------

show h225

show h245

show h323-ras

----------

SIP

-----
show sip

------
SCCP
-----
show skinny
------
I would not advise to get debugs at first - better leave those for interaction with TAC.
FYI - If it comes to TAC intervention, TAC will request "inside" and "outside" interfaces capture and debugs taken AT THE SAME TIME.

What goes without saying - !gathering packet capture helps!
If traffic is NATed and you see inside IP addresses in outside captures ... something went wrong. Maybe something similar to 4.?
Do you see packets coming into the appliance? Do you see them leaving? What has been changed.
FWSM note: Remember that inspected traffic goes straight to CPU!

Good luck.