One major factor in the emergence of agile methods in the 90s was the idea that we needed to manage software projects in harmony with the nature of making software. We fixed some of that, with shorter releases, iterating on plans; the emergence of the web as platform also helped.
But wasn’t agile rather about handling uncertainty and embracing change? Wasn’t it about any type of project facing uncertainty? Well, earlier efforts to get control over software projects were rather about removing uncertainty, deciding early on what shouldn’t change.
The belief was that late change was the very cause that projects failed – either exceeding budgets and deadlines, if finishing at all. We looked to other engineering disciplines in these efforts. Which was a mistake as it meant rejecting the nature of making software.
Also, having frequent, as in at least daily, discussions with those responsible for the behaviour and appearance of product as received by the users. You can’t successfully make software without this.
In some projects, UX becomes a barrier to such discussions. Those discussions have already taken place, and some artifact is handed over. Invariably there are aspects that haven’t been considered, usually leading to compromises in the resulting system design.