Large time behavior of nonlocal aggregation models with nonlinear diffusion