It’s because people’s wants have shifted as technology progresses. If everyone was satisfied to live like a medieval peasant and all we needed to produce was clean food/water we probably could have automated most of the agricultural work and done away with the need for the majority of labor.
But people today now want on-demand deliveries, entertainment, healthcare, telecommunications, international travel, etc. and they need to pay for these things somehow, which means work. These shifting desires continuously push the boundaries of what we are capable of producing which ends up redirecting labor rather than eliminating it.
Edit: thanks for the down votes everyone. I’m not saying this is the way it should be or that people should live like peasants, just explaining the basis of consumer/labor theory from economics 101. People typically get more utility out of the things they buy using their wages than they would from not working at all. Right now that’s mostly because society would let you starve to death, but even if there was UBI or something like it, there would always be some people who would want to work in order to buy more things for themselves.
I agree, the solution is not as simple as optimizing resource allocation. The problem is both social and economic in nature. The owners of production don’t just want money, they want power over others.