Novak Djokovic had no argument with Andre Agassi's blunt prediction