Since optimistic detection has advantages over pessimistic detection on case 3 & 4, is there any reason one wants to use pessimistic detection? What are the advantages of pessimistic detection over optimistic detection?

In cases with many restarts (like a more extreme Case 2 with more threads), there might be a significant amount of wasted work if we don't check pessimistically.

Does that mean pessimistic and optimistic detection may give out different results in certain cases? Given how they handle case 3 in different ways

I believe so @fdxmw. Having these two conflict detections means handling the conflicts in different ways, which are both reasonable to do. They have trade-offs among themselves, one of which is having fairness problems as you mentioned (see slide 50 on lecture 13).

As a note to self, in optimistic detection, we only check for conflicts are the transactions decide to commit as opposed to pessimistic detection where we check after every instruction

To understand this, I found it helpful to draw an analogy to cache coherence and bus transactions. A thread will shout across the bus only when it commits a transaction, and not when it performs a read or write. When a thread shouts across the bus, other threads will listen and take the appropriate steps to ensure atomicity.

We would only want to use optimistic detection if we believe there will be no conflicts. However, what if we do not know how many conflicts are going to happen. Is there a combination of both we could try?

lazy/eager differs in ways they write to memory (and roll back), optimistic/pessimistic differs in ways they check for conflicts

I'm a bit confused by how an optimistic read and pessimistic write would look in a diagram like this, as we had in the written assignment. Specifically, it seems like in STM, to validate a transaction (rd A) on a read commit, you would check if any of the other threads are writing (wr A), before commit by checking if there's a lock on A. But, in case 3, it seems like when T0 commits, T1 has the lock since it hasn't committed but T0 is successful. I can't tell where I'm wrong in my train of thought.

In optimistic detection, there's also the opportunity for starvation if one transaction is particularly slow and gets restarted over and over.