What exactly do we mean by "domain" here? What would be an example of a system being able to leverage domain knowledge to improve efficiency?

I think a simple example that was mentioned earlier in the lecture is PyTorch, a language that is written specifically for Deep Learning applications and designed to work on a specific data structure called tensor.

Examples of domain in my head include image processing (like real-time rendering for interactive games), graph computation or machine learning (as mentioned above).

I see that we can get a lot of improvement by losing generality and instead making the programming language intuitive and performant. However, it seems to me like generality isn't a huge benefit even if we did have it. If the goal we are optimizing for is solving some problem as efficiently / quickly as possible, what does it matter that the language we use can do a bunch of things outside the domain we need it for?

I see what you mean but I think that depends on how specific do we make our "domain". We need to remember that writing a whole optimized language is a very complicated and time-consuming process so we'd like to do it as less as possible and so we generalize our domain so it can achieve more things. Of course there's a balance. You don't wanna create another C++/Python if it already exists but you also might not be wanna be limited by C++/Python's implementation constraints.