How big is the GPU memory as compared to main memory in a typical machine? Does this mean that all data have to be copied from main memory to GPU memory first before arithmetic operations can take place? Does that introduce extra latency for loading data from memory for memory-intensive programs?

Before, the only way developers could run general parallel-computation programs on GPU's was through using programming constructs/syntax that were originally meant for graphics. The key idea here is the development of a new interface that programmers could use to run general parallel-computation programs on GPU's that didn't necessarily involve graphics.

New Compute Mode to allow for general purpose computing on GPU leveraging Single Program Multiple Data (SPMD) model

@laimagineCS149 GPU tends to have around a comparable amount to CPU RAM. I think one of the goals for GPU intensive computation is to reduce a lot of memory copying overhead.