This may be a dumb question, but why triangles? Is it just because that adds more flexibility into what we can represent?

It might be because any polygons could be divided into triangles, and therefore offers the flexibility for representation.

I think it's also relatively efficient to do computations on a triangle (e.g., it is efficient to compute whether a point is inside or outside a triangle). And as @krisinayige mentioned, any polygon can be represented by a collection of triangles. So in your hardware and software, you can really heavily optimize for operations that assume triangles => more efficient computation (much in the same way that RISC-style instruction sets are preferable to older instruction sets like x86 since optimizing your hardware to run only a few instructions actually gives you better performance in the general case than trying to handle many instructions).

To add, a triangle is the most simple representation of a plane in 3D

Are there cases where using a polygon (say square) might be better suited than a triangle?