Your backbone access to the Internet is shared when you are using a cable modem. Meaning that multiple users are on the same node. You are probably experiencing network congestion during peak hours due the the high number of users on at the same time. It is also possible that someone on your network has manipulated their modem to use more bandwidth than they have been allocated. In which case you will need to continue to report the slow connection speeds to your ISP so that they can more closely monitor that node for misuse.

good luck.