In class, we pointed out that communication within a rack is faster than communication between racks.

How do we load balance the bandwidth between the ToRs?

So communication between nodes in other racks is slower than communication between nodes over the network...?

@hamood, I believe the "Network" here refers to the connection between nodes in a rack to the rack switch. Basically what this slide is saying is that data locality is very important (note up to 1000x difference in GB/s from nodes in other racks to CPU->DRAM), so it's worth preserving data locality in your system as much as possible (e.g., fusing operations in Spark).

This slide was super helpful for me in understanding the orders of magnitude at play. Data locality is clearly super important in an environment like this!

This probably sounds dumb but I got a bit confused on why the SSDs bandwidth is much smaller than the DRAM while the DRAM's storage is smaller since I've read before that SSDs are much faster than HDDs but then I realized that DRAM is not an HDD and in fact refers to the RAM in a device (hence D"RAM"). So a DRAM is what our devices use to execute operations quickly which is why it's much smaller and has a much higher bandwidth than our SSD. While SSDs are where we store our data and we want to avoid loading data from it repeatedly which is why it has a much lower bandwidth.