On each of the routers, assign a static DHCP address based on the MAC address of the other person's wireless card. Based on that address, you should be able to restrict that machine from getting to the internet within the router config.
Unfortunately, most wireless cards only allow you to be on one wireless network at a time, so you would have to switch between the two. Each network would need to have its own SSID.
Now, if you wanted to get fancy, you might be able to pull something like this off:
-Connect your routers with an ethernet cable. If this is not possible, then this plan won't work.
-Make sure your routers have different internal IP addresses. For example, set one to 192.168.0.1 and the other to 192.168.0.2
-If yours is .1 set your default gateway to 192.168.0.1. Then set their default gateway to 192.168.0.2.
Unfortunately, you will experience difficulties if you set your SSIDs to the same name, and you do not have a connected LAN behind them.
I've never done this, but I suppose it would work pretty slick if you had 2 wireless cards in each of your computers, one on each of your own networks and one on each other's.