I am a bit confused what we mean by program order (is this for a single program or several programs being executed in parallel).If it is a single thread and a single program this makes sense, but wanted to clarify.

I am wondering why are we trying to enforce the memory operation ordering? Shouldn't programmer synch the access to shared memory?

@martigp I believe this is in reference to the program order of a single thread/single program. Even in this case, the computer can reorder operations (in order to hide the latency of a write, for example). It seems that there is a balance between the hardware enforcing strict adherence to the software's sequential outline of operations, and the hardware behaving in a more relaxed way when it comes to always performing operations in sequence, potentially placing more of a burden on programmers to ensure correct program behavior (e.g. adding synchronization/barriers to get rid of race conditions/unexpected behavior).