Newer routers have expanded 'buffer' sizes, which actually works against the TCP protocol.
Because they hold many more packets, and don't 'tell' the ISP servers to slow down the feed of packets until they get full, it just creates more problems in terms of latency...despite having large sizes of feeds (Mbps) to and from the ISP.
All the computers on the network have equal access to the bandwidth... they're all able to put packets onto the network that'll go out through the router at the same time.
By deliberately limiting the router to 600Kb buffer, everybody else in the family can stay interactive, by not letting the buffer get too overloaded with qued packets.
(Condensed from GRC-Security Now; https://www.grc.com/sn/sn-345.htm & https://www.grc.com/sn/sn-348.htm
Test all your relevant network stats, but pay particular attention to "network buffer measurements" to see what it is:
ICSI Netalyzr: http://netalyzr.icsi.berkeley.edu/
Requires Java, NoScript users can 'temporarily allow berkley...'
The results will give the entire Network, which includes your ISP, so although your router may be just fine, it could be the ISP config's that are mis-adjusted or could be tweaked. Many's the time a person complains to the ISP, and they flatly deny any responsibility, without actually examining their configs.
Portions of this podcast explain 'buffer bloat' and it sounds a bit like what you have.
Using the "Find" function (ctrl + F) with this phrase "the effect of the buffer bloat" will take you to the question/answer containing the explanation.
https://www.grc.com/sn/sn-348.htm