Why an external anchor exists
Without an external reference, a large enough participant could move the venue's own price and trigger liquidations against others. The oracle makes that attack far more expensive.
It also means the perp can visibly deviate from the oracle for a period — that deviation is what funding is charged on.