Speaking At OpenWest 2014

OpenWest Logo
I will be at the 2014 OpenWest conference with a presentation titled "Secure Your VoIP with Asterisk and Kamailio". Looks like they have me penciled in for 3pm on May 9th. Watch the schedule for changes and take a look at all the other awesome talks as well. I've attended and presented a number of times at OpenWest (aka UTOSC) and it's always a great venue.


Kamailio Hashtables

A few weeks ago I completed an upgrade from OpenSER 1.2 to Kamailio1.5. Overall it's been working quite well and actually was not nearly as hard as I feared it might be. Now that I've got this new version with a ton of new bells and whistles, I thought I'd try out a few of them. One of the most exciting is shared memory hashtables.

First a quick primer on hashtables. Currently whenever the proxy needs a value from the database it simply queries the database and gets the value. Straightforward, right? But as the system grows, the database can be greatly put under strain. In my Kamailio cluster the single biggest CPU user is in fact the MySQL database by a large margin. And since the database changes pretty infrequently it would be nice if we could cache those values.

The problem is that Kamailio is a multi-process daemon, so what's cached in one isn't going to be cached in another. Enter shared memory hashtables. The same cache is shared among all the processes. Usage is pretty simple. Here's an example:

# IP based authentication
if ($sht(ht1800=>$var(fU)::ipauth) == null){
	if (is_user_in("From", "ipauth")){
		$sht(ht1800=>$var(fU)::ipauth) = "yes";
		$sht(ht1800=>$var(fU)::ipauth) = "no";
	$var(ipauth) = $sht(ht1800=>$var(fU)::ipauth);
	$var(ipauth) = $sht(ht1800=>$var(fU)::ipauth);

if ($var(ipauth) == "yes"){
	# do whatever...

Don't worry too much about the nitty-gritty. First we check to see if the value is cached (($sht(ht1800=>$var(fU)::ipauth) == null)). If it's set, we just use the cached value and continue. If not, we have to ask the database (is_user_in("From", "ipauth")) and then save the result in the cache. It actually is that easy.

But the $64,000 question, has it helped? Luckily I thought ahead and got some data before I made the change. I grabbed the query log for a day before and after the change. I made sure to select two days where the call usage was the similar (1% difference). Overall there was a 20% reduction in SQL queries.

I only put in the one change as a proof of concept. Since it sure seems to have panned out I'll continue wrapping all my database calls with hashtable caches. One thing I still need to work out is how to invalidate the cache when I make a change through my provisioning system. And once I'm able to upgrade to Kamailio 3.0 I'll switch over to memcached, which has all the benefits of a shared memory hashtable but it's also clustered. Ooh, I'm so excited!


VoIP Routing Loop

The other day my boss and I managed to create a call routing loop between two of our phone systems. It was kinda fun, in one of those painful ways. Here's how it went down.

Phone system A has extensions in the range of 2XXX. Phone system B has extensions in the rage of 1XXX. The PBXes have an IAX2 trunk between them to allow direct dialing. The phones themselves have a dialplan which recognizes 1XXX and 2XXX as patterns, which allows for faster dialing. Users must dial a 9 to reach an outside line.

There was just a slight problem. Due to a misconfiguration on my part, B was set up to route any unknown 1XXX extensions back over to A. For a real extension, say 1000, the rule wouldn't match because B would know to send that call to the local user. But let's say that somebody's boss forgot to dial a 9 when dialing a long distance number. So the phone sees the first 4 digits, 1 + area code, and assumes it's an extension. A sends the call to B. Well, that extension didn't exist so B sent the call back to A. That would be the point when the fun began.

I would have thought that the PBXes would create a whirlwind of calls until the original call eventually timed out and broke the chain, thus bringing all the calls to a halt. But somehow that isn't what happened. The system stabilized at 400 or so active channels. Sat that way for a few hours I believe. What finally did stop it was my removing the rule to allow 1XXX over the IAX2 trunk from B to A. Within just a few seconds all the calls came to a grinding halt (in a good way).

I've decided then that having local extensions that start with a 1 is a less than desirable thing. It's not the end of the world, but avoiding it can relieve some potential headaches so I intend to eschew it where possible.


VoIP QoS With Wondershaper

Hans and I were discussing QoS the other day, specifically regarding using Wondershaper from the LARTC. I had managed to mess mine up and I subsequently noticed a horrible turn for the worse in my VoIP calls. Wondershaper has to be adapted for use by OpenWRT and in the process I misspelled sch_ingress.o as sch_insmod.o. Too much insmodding that day, I think. The net effect was that download speeds were not shaped at all.

Once I got it corrected, I decided to do a few tests just to confirm that using Wondershaper actually made a difference. I'll cut to the chase for the lazy: it did. I made 45 second calls to music on hold from my softphone, Twinkle. In the background I had Wireshark running. I used the RTP analyzer in Wireshark to look at the statistics after all was said and done. I used both versions of Wondershaper, the CBQ and HTB. I had a single download running the whole time eating up all spare bandwidth.

With no shaping: 4.4% loss (95 packets), 60ms jitter
With CBQ Wondershaper: 0.2% loss (5 packets), 35ms jitter
With HTB Wondershaper: 0.3% loss (6 packets), 28ms jitter

So my unscientific conclusion is that both versions of Wondershaper work about the same and they both make a huge difference. I could easily hear the packet loss on the first call, but not so much on the other two calls.



My bud Hans and I tonight tested out encrypted VoIP with ZRTP. I noticed a while back that Twinkle supports it and have wanted to test it out, but none of my desk phones support ZRTP.

It was fun. When the call terminated, Twinkle displayed a cute message about verifying the SAS (short authentication string). It was 4 character (hprj, if you're curious) that represented our encryption key. It's the way ZRTP verifies that a man-in-the-middle attack is not underway. There was a padlock icon which we both clicked to verify that the SAS was correct. I'm not sure what if anything happened because of that, except that we both verified that our SIP phones have not been tapped by the feds.

In the SDP, ZRTP is advertised with "a=zrtp". It's not a separate protocol per se. The actual codec was selected through the normal means (we used speex/16000). Looking at the RTP data, I see a whole bunch of "AES256", "SHA256" and "DH4096". Presumably that's part of the ZRTP negotiation. I didn't delve further. What I see though is that the encrypted data is simply represented as Speex RTP, but the actual data has been scrambled so it would be meaningless to a passerby.

Based on this testing, I predict good things for ZRTP. It was quite painless to use as a caller. As long as it's enabled by default in the phone, there's really nothing else that a user has to do to use it. The SAS is short and you only have to verify it if you care. Phil Zimmerman says that you don't even have to verify the SAS every time. Just once in a while is good enough. And obviously anytime you're conducting private business (which is not the same thing as illegal business). The simple fact that ZRTP is used every time means that you can't tell whether a call is valuable or not just based on it being encrypted.

The one possible failure of ZRTP is that it doesn't hide any of the signalling data, so a spy would be able to see who you were calling. That problem would be quite hard to solve. I'm not sure of the benefit either as the cost to mask that information is much higher. You pretty much have to know all the routing information ahead of time. Even then, an eavesdropper could still see the two IP addresses involved, which will give away some amount of information. So for now, ZRTP is a good solution.



I had the dubious honor the last couple days of breaking up an argument between a Linksys PAP2T and a Linksys WRT54G. You might think they would work well together, coming from the same family and all. Well not really.

The problem was caused by the WRT54G (version 5, btw) which refused to route SIP traffic back to the PAP2T. It was allowing HTTP and DNS traffic just fine, but it balked at SIP for some reason. The symptoms were that the PAP2T kept sending REGISTER requests without a WWW-Authenticate header. For a long time I thought it was a setting on the PAP2T, but instead it was because the PAP2T never got the WWW-Authenticate challenge, so it obviously couldn't respond. A dead giveaway should have been that the PAP2T would send 5 REGISTER requests in a row. It must have figured there was network congestion or something, so it was retransmitting.

The solution was to upgrade the WRT54G from version 1.00.0 to 1.02.2. Some bug in the 1.00.0 firmware caused the issue and now it's resolved. I haven't looked at the release notes to see what it was, and frankly I probably won't. I've never really liked the version 5 WRT54Gs and this just helps solidify it.


Subscribe to RSS - voip Subscribe to zmonkey.org - All comments