One major reason why the "free lunch" is coming to an end is that we've hit heat and power constraints when trying to scale up processor frequency. Even if you could cool chips as you continued to increase processor frequency, most of your costs would go towards the cooling process (diminishing returns on scaling frequency).

If there is some method of efficiently cooling down the system; for example, redesign the hardware layout such that the heat could be let out more easily, would the bottleneck of processor clock rate be possibly removed? Are we more concerned about heat dissipation or the consumption of power?

Does the "power wall" become less of an issue when working with multiple processors? Will there eventually be limitations in the future when working with more and more processors (i.e. something similar to the "power wall")?

I've heard that some chips can run hot enough that they'll have to move their computations around, so that the same regions of transistors aren't constantly producing heat. At the same time, it sounds like the computations that are required to determine whether a given region of the chip is in danger of overheating are themselves responsible for releasing more heat, which might result in something of a feedback loop. I'm wondering how this heat regulation problem is managed in practice, when the process of collecting/analyzing temperature data actually seems to make the problem worse.

@kristinayige, based on my knowledge, dissipating heat is a really difficult problem. Given that the whole industry uses silicon to make chips, we are rather limited by the choice of the materials. At the same time, given that we are trying to keep the chips small to avoid longer wire latencies, we can't really increase the area of the chips to make the dissipation of heat easier. I think the main efforts to solving overheating are in architecture and in physical design. RISC processors are known to be much more power-efficient than CISC processors, which is why M1 macs are able to deliver such a ridiculous amount of performance by only consuming 13W of power.

One way 'around' this power wall is, if you have enough transistors available, adding specialized HW for certain operations, like decoding/encoding video and turning 'off' those blocks when they are not in use. Using specialized HW uses less energy than general purpose HW, allowing you to use more transistors without increasing the power dramatically. This pertains more to the mobile space than the server space but this is still important.

@gklimias, to my knowledge, the main reason why we can't increase the physical area of chips is cost. Usually, you pay the fab a fixed cost per wafer so increasing the chip area affects the number of chips you get per wafer, driving up the per chip cost.