The owners of a heritage mansion in one of Brisbane’s most exclusive streets have given up their battle with a developer building a luxury high-rise next door.