Can you explain why we need more tasks than processors?

If there are more processors than tasks, some processors will be idle. Processors can fetch another task if they are done.

A common question that was asked during lecture was if we should have more tasks than processors. A useful follow-up question that Kayvon asked in lecture was, "what must be true about the tasks for us to only use the same number of tasks as the number of processors we have?" The answer to this question is that these tasks must be of equal cost to prevent processors from going idle. I think this slide does a great job summarizing why having more tasks than processors is helpful. Having more tasks helps us use a dynamic workload balancing style to ensure our processors are performing at maximum utilization.

If we have a sufficiently large number of granular independent tasks, then it seems like by the Law of Large Numbers if we assign each processor to do the same number of tasks and partition the tasks at random, all the processors should finish at roughly the same time. (Importantly, all the task to processor assignment can be completed before starting any of the tasks.) In this case we don't need to do any fancy logic for scheduling tasks, so in theory we should make tasks as granular as possible with the negligible added overhead. Is there a reason that this approach isn't used in practice?