there are no references at all, that point to a ddos attack. ssh is protected by fail2ban i also dont use a standard port. also the rest of the server is very responsive.
i also tryed free && sync && echo 3 > /proc/sys/vm/drop_caches && free in case its a memory problem, but it did not help.
@mehargags, i tryed your suggestion with disabling some sites. Problem is, which i forgot, apache 2 restarts after i suspend a site. As it takes a long time for the server to get slow again i cant really test it, as i cant leave the sites disabled for to long :(
Maybe its also worth mentioning that the CP itself is still blazing fast, even without a restart.