Although WiFi is convenient for mobility, wireless interference can cause it's connection speed and quality to vary wildly.
Wireless signals are affected by many factors including distance, wall density, electrical interference, directional antenna range, etc.
All of these factors will affect your actual wireless range. As wireless connection quality varies the connection speeds are renegotiated. The WiFi connection between your device and the wireless router may start at 300 Mbps and end up at less than 10 Mbps or drop entirely.
You should try to centrally locate your wireless router or gateway up high and away from dense walls and electrical and 2.4 GHz wireless interference such as microwave ovens, cordless phones, baby monitors, etc.
For serious gaming or video streaming your best bet is a direct Ethernet connection. Ethernet is unaffected by wireless interference and will give you a rock solid, stable network connection.
That being said,
I would have guessed WiFi drivers, however you mentioned that you tried another adapter, which would have used a different driver.
Since you mentioned that the problem occurs at specific times, electrical interference would be my next guess. As you have noticed, a Ethernet cable will resolve WiFi interference problems.
If it is not feasible to run Ethernet cable your next best option is to use powerline network adapters. I wrote a review here: http://isp1.us/reviews/netgear-xavb2101-powerline-adapter/
You plug one in by your router and one in the room where you need access and connect them to the devices with standard Ethernet patch cables. The network signals are transmitted over your existing home electrical wiring which will not be affected by wireless interference. There is still overhead in the connection but it is much more stable than WiFi.
If you decide to replace the router.
It pretty much comes down to what features you want and how much you want to spend.
The router brand is not as important as the model. Some manufacturers release a good model but the next is crap. I would look at the Amazon reviews of models that interest you. Even if you don't buy from Amazon the reviews will tell you a lot about which models to avoid. Look for at least 15 or more reviews with a 4 star or better rating.
For under $50, this Medialink is the most popular and highly reviewed model. http://www.amazon.com/gp/product/B00A3YN0Z0/ref=as_li_ss_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=B00A3YN0Z0&linkCode=as2&tag=excharge-20
If you want to spend a little more, many quality dual band wireless routers are available. http://www.amazon.com/s/?_encoding=UTF8&camp=1789&creative=390957&keywords=dual%20band%20router&linkCode=ur2&qid=1374595143&rh=n%3A172282%2Cn%3A541966%2Cn%3A300189%2Ck%3Adual%20band%20router%2Cp_72%3A1248879011&rnid=1248877011&tag=excharge-20
Dual band routers have multiple wireless radios enabling broadcast on the 2.4 GHz and 5 GHz bands simultaneously. Using the 5 GHz band cuts down on interference from cordless phones and microwave ovens.. Not all devices can use the 5 GHz frequency. Many laptops and other devices only operate at 2.4 GHz. Thus the need for dual band routers.
Good Luck...