Heat the oil in a 3-quart pot.
Add the shallot and scallions and cook over medium heat for 1 minute.
Add the broccoli and vegetable broth.
Cover and cook until the broccoli is tender, about 15 minutes.
Pour the broccoli mixture into a blender and puree.
Place the broccoli mixture in a pitcher.
Add the milk, cream, pepper, salt, and nutmeg.
Chill the soup from 2 to 24 hours.
Serve cold.