You can't access the Internet on more than 1 device at a time without the use of a router.
Here is how it works,
A router connects two separate networks and manages the traffic between them. In most home network applications the router connects your local group of computers and devices known as the LAN with the Internet, commonly called the WAN or Wide Area Network.
The router enables you to connect multiple devices together into a home network (LAN), and share a single ISP (WAN) Internet connection, so all of your devices can access the Internet.
Without a router to manage sharing of the WAN connection, your home network devices can share files and resources via the router's LAN connections, but only one device at a time can access the Internet.
All you need to do is connect your existing switch to one of the multiple LAN ports on the router. There is a WAN port on the router that connects to your ISP Internet connection.
The switch manages the traffic between the devices and the router and the router manages DHCP (IP addressing) and WAN (Internet) traffic.
You can turn off DHCP on the router if you like and set all iP addresses manually. The devices and the router all need to be in the same subnet though.
For example,
Router IP 192.168.1.1
PC 1 IP 192.168.1.2
PC 2 IP 192.168.1.3
PC 3 IP 192.168.1.4
etc...
That being said,
Most routers sold nowadays are wireless models. If you don't need or want the WiFi functionality, it can be disabled in the router configuration.
Wireless routers are relatively inexpensive,
A wireless router enables you to provide simultaneous wireless (WiFi) Internet access to all of your WiFi enabled devices and also all cabled (Ethernet) devices.
It pretty much comes down to what features you want and how much you want to spend.
The router brand is not as important as the model. Some manufacturers release a good model but the next is crap. I would look at the Amazon reviews of models that interest you. Even if you don't buy from Amazon the reviews will tell you a lot about which models to avoid. Look for at least 15 or more reviews with a 4 star or better rating.
For under $50, this Medialink is the most popular and highly reviewed budget wireless router model on Amazon. http://www.amazon.com/gp/product/B00A3YN0Z0/ref=as_li_ss_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=B00A3YN0Z0&linkCode=as2&tag=excharge-20
Good Luck...