Mike Ferrier

I beat code into submission.

Using Nmap and Socat to Get Around Public Internet Port Restrictions

In a previous post, I detailed how I set up a VPN server so that I could internet securely while traveling and using public internet access points. Public internet is convenient, but is usually insecure by default, so tunnelling all your traffic through a VPN is a smart bet.

However, some public networks can be a bit restrictive with what kinds of traffic they allow. For example, when we were staying in Fukuoka we stayed at the interestingly-named Hotel Active! which, though it was a great hotel and had free internet, would only allow you to send traffic out on a few ports. I had a sneaking suspicion going in that the free internet might give me problems as I had read other reviews of this hotel chain that suggested it might be troublesome.

Not only would this mean that I couldn’t use my secure VPN, but it would also prevent me from using SSH to connect to Github and other work-related secure connections. Annoying!

Thankfully there are ways around these restrictions. If they allow any outgoing traffic on any port, then you can run a server remotely to receive that data and relay it to your VPN server. To do this, we’re going to run nmap to figure out which outgoing port we can use to connect, then we’re going to run socat on our VPN server on that port, and relay the traffic to our real VPN server.

  1. Use nmap to figure out whitelisted ports
  2. Run socat to relay traffic to your VPN server
  3. Update your VPN client to use the relay port
  4. Troubleshooting

1. Use nmap to figure out whitelisted ports

The first thing you have to do is figure out on which ports the network will allow outgoing data. For this you can use the excellent nmap security scanner. Once you’re on the restrictive network you can use it to scan a bunch of regular ports and see which ones are allowed through and which ones aren’t.

WARNING: While port scanning is an invaluable tool for debugging and troubleshooting networks, it can also raise suspicion of malicious activity. For example, port scanning for a vulnerable piece of software with the intention of exploiting it. Suspicious port scanning can get you in trouble with your ISP or network administrator, so use it sparingly and if you’re not sure if you know what you’re doing, don’t do it at all.

To install nmap on OSX, you can just use homebrew and run:

1
$ brew install nmap

Once you have nmap installed, you’re ready to start scanning.

On an unfiltered network, the results of the scan will show open and closed ports, like this example where we probe ports 75 to 85:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
$ nmap mikeferrier.com -p 75-85 -Pn --reason

Starting Nmap 6.25 ( http://nmap.org ) at 2013-07-14 14:44 EDT
PORT   STATE  SERVICE    REASON
75/tcp closed priv-dial  conn-refused
76/tcp closed deos       conn-refused
77/tcp closed priv-rje   conn-refused
78/tcp closed unknown    conn-refused
79/tcp closed finger     conn-refused
80/tcp open   http       syn-ack
81/tcp closed hosts2-ns  conn-refused
82/tcp closed xfer       conn-refused
83/tcp closed mit-ml-dev conn-refused
84/tcp closed ctf        conn-refused
85/tcp closed mit-ml-dev conn-refused

The extra argument -Pn tells nmap not to ping the host, just to scan it, and —reason prints out the reason the port state was resolved to the value shown. This will come in handy later.

However, on a filtered network you’ll usually be able to see which ports are filtered. In this example, I’ve manually filtered outgoing ports 75-80 to show what it looks like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
$ nmap mikeferrier.com -p 75-85 -Pn --reason

Starting Nmap 6.25 ( http://nmap.org ) at 2013-07-14 14:44 EDT
PORT   STATE    SERVICE    REASON
75/tcp filtered priv-dial  no-response
76/tcp filtered deos       no-response
77/tcp filtered priv-rje   no-response
78/tcp filtered unknown    no-response
79/tcp filtered finger     no-response
80/tcp filtered http       no-response
81/tcp closed   hosts2-ns  conn-refused
82/tcp closed   xfer       conn-refused
83/tcp closed   mit-ml-dev conn-refused
84/tcp closed   ctf        conn-refused
85/tcp closed   mit-ml-dev conn-refused

You can see that nmap got “no response” from ports 75-80, and so marked them as “filtered.” Depending on how the firewall is configured, sometimes instead of “filtered” you’ll see filtered ports marked as “closed” but the reason will be “reset,” which is a different way for firewalls to deny traffic but amounts to the same thing: you can’t send traffic on those ports.

What you’re looking for here is an outgoing port that isn’t filtered, so that you can use it to send out all your tunneled VPN traffic and bypass the firewall. When I scanned from the hotel, I noticed every port from 1-100 was filtered except 53 (DNS), 67 (DHCPS), and 80 (HTTP). It looked something like this:

1
2
3
4
5
6
7
8
9
$ nmap mikeferrier.com -p 1-100 -Pn --reason

Starting Nmap 6.25 ( http://nmap.org ) at 2013-07-14 14:54 EDT
Not shown: 97 filtered ports
Reason: 97 no-response
PORT   STATE   SERVICE REASON
53/tcp closed  domain  conn-refused
67/tcp closed  dhcps   conn-refused
80/tcp open    http    syn-ack

So 97 ports were filtered, one was open (web), and two were closed, but responding as closed. This will clue you in that traffic to these ports are allowed out by the network. For my purposes, I chose port 67 to work with.

2. Run socat to relay traffic to your VPN server

socat is an excellent multipurpose relay tool. It can pretty much relay traffic from anywhere to anywhere, and so it’s the perfect choice for relaying our traffic from the unrestricted port to our VPN server. Installing socat is left as an exercise for the reader, but you need to install it on a remote server, ideally on the same server as your VPN server so that you’re relaying locally and not across the internet.

I’m using port 67 as the relay port, and my VPN server listens on UDP port 1194, so you need to configure socat to listen for UDP packets on port 67, and relay them to localhost UDP port 1194:

1
$ sudo socat UDP-LISTEN:67,fork UDP:localhost:1194

The fork directive tells socat to fork a process for each connection so that you can keep listening on port 67. Without this, socat will terminate along with the end of the first connection.

You won’t see any output from this command, but if you want to troubleshoot or verify it’s working, you can add a -v flag to make socat’s output more verbose. It’ll spit out a ton of garbage when you connect so you can tell the connection is being made.

3. Update your VPN client to use the relay port

Once this relay is set up, you’re ready to connect. You have to reconfigure your VPN client to use the new relay port, though. This is generally done either through a configuration GUI or by editing a config file.

I use the very intuitive Tunnelblick OpenVPN frontend GUI for OSX, so for me the process is:

  1. From the Tunnelblick config screen, select your VPN configuration and click the gear icon, and select Edit OpenVPN Configuration File

  2. Find where the port is specified, and switch it to your relay port:

Once that’s done and saved, connect to your VPN and you should be up and running.

4. Troubleshooting

If for some reason it doesn’t work, you can troubleshoot by running nmap on the relay port to make sure you can connect to it from your local machine:

$ sudo nmap mikeferrier.com -p 67 -sU -Pn —reason

1
2

You can also start <code>socat</code> in verbose mode so that any connection activity is output to the screen:

$ sudo socat -v UDP-LISTEN:67,fork UDP:localhost:1194 “`

Happy relaying!

Living in Japan: the Cellphone Situation

I love Tokyo, but it can be a nightmare to navigate. It has the same problem that any ancient city has, which is that the people laying out the streets made things up as they went along.

Compare Manhattan’s carefully planned and easy to use grid system…

… with Tokyo’s sprawling labyrinth of streets:

The upshot of this is that, when in Tokyo, bring a GPS. Luckily, just about everyone now has a GPS-enabled map in their pocket on their smartphone. So all you need to do is bring an unlocked phone, get a data plan, and you’ll be good to go.

As for which cellphone provider to use, there are plenty of MNOs and MVNOs to choose from, but the problem with the big three MNOs (Docomo, KDDI, and SoftBank) is that they all want you to buy a new handset from them, enter into a multi-year contract, and buy full package Voice + SMS + Data plans. Blech.

In this post I’ll be focusing on the excellent MVNO B-Mobile as they seem to offer the best prices on both short- and long-term data plans.

I’ve found MVNO data plans generally come in two flavors:

  • Low speed (usually 30kB/s or less) – good for things like email and data-based messaging, not-so-good for data heavy stuff like web. Also explicitly blocks “streamed” services like music, video, and Skype.
  • High speed (3G/4G/LTE speeds) – as fast as the network you’re on can go, and usually allows streaming services (check the fine print).

If you’re staying in Japan for a month or less, it’s generally a good idea to just get a visitor’s prepaid SIM card. That way you can sidestep the paperwork that goes with getting a longer term plan. Trust me, it’s just easier.

For prepaid SIM cards, 1 GB of highspeed will run you about 4,000 yen or you can get 14 days of unlimited low speed for the same price — info here. Heads up though, the 1 GB of data expires 14 days from the day it is shipped, so if you screw up on the address or can’t get it working, the clock’s still ticking. An alternative to consider is Econnect Japan. Their 1 GB prepaid plan is slightly better than B-Mobile’s as it’s the same price but your prepaid data lasts 30 days instead of B-Mobile’s 14 days.

Compared to long-term data plans, prepaid data plans are kind of expensive. So if you’re going to be here for a month or more, you’re going to want a recurring subscription data plan. There’s one caveat, though: there are government regulations on these longer-term plans in order to curb cellphone fraud. Basically, providers need to verify your address through a call from you to them on a Japanese cellphone or landline, or through them mailing you a confirmation code to a Japanese residential address. So if you’re staying in a hotel during your stay, you may have to employ some creativity (e.g. using your hotel’s landline) to fulfill the requirements.

For long-term plans, B-Mobile seems to have the best prices and widest variety of deals right now. Since my wife and I both wanted data we opted for the PairGB SIM which is kind of a great deal: you buy the two SIM cards for 3,150 yen and then you sign up for a monthly subscription: 2,970 yen for 2 GB between both SIM cards per month. $15 per month for a GB of data with no contract? That’s a good deal no matter what country you’re in.

You have three options for purchasing:

I went with the Yodobashi option. Make sure you get the appropriate SIM format for whichever phone you have. We each have an iPhone 4 so we got the Paid GB Micro SIM format (“マイクロ” is Japanese for “Micro”).

Eventually, you’ll have the B-Mobile SIM card(s) grasped firmly in your hot little hands. They’ll actually be Docomo SIM cards as that’s who B-Mobile is reselling, and they’ll look something like this:

That silver thing is the SIM card pop-out pin that you got with your iPhone. You did remember to bring it with you, right? Don’t worry, an unbent paper clip will work too.

Now simply follow the instructions inside the package and go to the appropriate activation URL. Do your best to stumble your way through the Japanese forms (the automatic page translation in Chrome helps a ton) and when you finally finish, it tells you at what time the cards will be activated (about 45 minutes from the completion of the form in our case) and also that a confirmation code is being mailed to your address.

Now you can put the SIM cards into your phone. Be sure to enter the correct APN settings for the product you bought, as they’re different for each product. A list can be found here.

A good habit to get into is to reset your “cellular usage” meter each month so that you can keep tabs on how much data you use. You can get there in iOS from the Settings app –> General –> Usage –> Cellular Usage. It looks like this:

The logged-in area of B-Mobile’s website will also tell you how much you’ve used up so far in the month. Also there you will find how many days left you have to enter the confirmation code. You remember the confirmation code, don’t you? The one they mailed you? If you don’t enter it in 30 days, they’ll cancel your subscription.

A word of warning: my SIM cards didn’t work out of the box for some reason, and I had to call the B-Mobile English help line a bunch of times to get it fixed. By some separate but equally mysterious glitch I couldn’t get through to that line through Skype, so I had to use Melissa’s Japanese crap phone. Considering her phone was like $20 and it saved my ass, I’d also recommend getting one if you’re spending more than a month here. A post on that coming up soon.

Hope that helps you out, and feel free to ask questions in the comments.

Living in Japan

Melissa and I decided last year to spend the first 6 months of 2013 living in Japan. We’ve both always wanted to spend some time living abroad, and it just so happens that, at this point in our lives, all the stars seemed to align at once to provide this opportunity. We’re both able to work remotely, our condo lease was up, we’re both eligible for the Working Holiday Visa, so we figured it was time.

We’ve finally made it here and settled in, and after a crazy first two weeks I decided I should start posting about the things we’ve had to figure out that might be useful to others who come here to live as well.

While we’re here, we’re going to be pretty much exclusively using AirBnB for accomodations, which is significant: without AirBnB this excursion would have been much more difficult and expensive. Before AirBnB, renting a place in Japan involved such unpleasantries as

  • hiring an agent to find rental properties
  • negotiating with a prospective landlord in broken Japanese
  • paying 2-3 months rent in refundable deposit
  • paying 1-2 months key money in unrefundable “key money”

Key money is one of those things about Japan that, coming from other places in the world, blow your mind when it’s first explained to you. Basically it’s a gift of 1-2 months’ rent to the landlord. That’s right, not a deposit but a gift. For the privilege of allowing you to rent from them. Fucking bonkers.

Apparently it’s a practice that dates from the end of WW2 when rebuilding efforts were still being undertaken and housing was scarce. More info here.

AirBnB listings for Tokyo have been slowly growing this year, but there are currently around 150 listings which is a pretty healthy selection to choose from. With AirBnB you can book long-term accomodations and pay much less than hotels — the place we’re living in was around $60/day.

One thing to remember if you go this route is that gift giving is an important part of the social niceties that are expected of the Japanese. Be sure to bring or buy gifts for whoever you’re renting from to show them you’re a thoughtful foreigner and not a bum.

Rescuing Multiple Exception Types in Ruby and Binding to Local Variable

Took me a few minutes to figure this out and wasn’t easy to Google, so hopefully this helps someone out.

Rescuing multiple exceptions in one rescue clause is pretty intuitive:

1
2
3
4
5
begin
  rand(2) == 0 ? ([] + '') : (foo)
rescue TypeError, NameError
  puts "oops"
end

I wanted to also bind the exception, whatever it is, to a local variable. To do that for a single exception is like:

1
2
3
4
5
begin
  [] + ''
rescue TypeError => e
  puts "oops: #{e.message}"
end

To combine the two, list the exceptions and then name the local variable with the last type in the list:

1
2
3
4
5
begin
  rand(2) == 0 ? ([] + '') : (foo)
rescue TypeError, NameError => e
  puts "oops: #{e.message}"
end

Using Google Latitude to Map Your Travel

Melissa and I just got back from our mini-rtw trip to Asia, during which we had data roaming on our phones. I kept the Google Latitude app open in the background, which pings your location to Google every so often.

When I got back and took a look at the map, the results were pretty cool. Some highlights:

Here’s us going all over the place in Tokyo, with our home base in Shibuya. You can see the Yamanote Line loop mapped out clearly. You can also see our trip to the Imperial Palace in Chiyoda for the Emperor’s New Year’s address.

During our Hong Kong stop we took the Turbo Jetfoil to Macau — you can see the ferry’s path above.

In Hong Kong we stayed in Aberdeen, which is on the south side of Hong Kong island. Every day we would take a taxi up to the north side which would cost about $60 HKD, or around $8 CAD. As you can see, sometimes the cab took the toll tunnel, other times the winding mountain roads. Also apparently we hung out in Wan Chai and Causeway Bay a lot.

Here’s the entire trip if you’d like to play around with it:


View Asia Trip 2011 in a larger map