Property sales in NSW since 1990

The NSW Valuer General publishes every recorded property sale in the state as free bulk data, several million transactions going back to 1990. I’ve started a project based on this dataset: NSW Property Insights, to see what interesting insights can be generated. This is the first of a series of posts on working with this data.

Sale records per contract year, coloured by the archive format each record came from. The years around 2000–2001 show both colours because a sale contracted near the boundary can turn up in both archives: lodged late, it lands in the newer format as well as the older one.

What the data you download looks like

The data arrives as semicolon-delimited .DAT files, zipped by year, in three archive formats that have changed over time:

Era	Format
1990–2001	“Archived” yearly zips, old record layout
2001–2014	Yearly zips with nested directories
2015–present	Yearly zips containing weekly zips

Each row starts with a single letter marking its type: A is a header, B is a sale (the property, its address, price and dates), C is a legal description, D is suppressed owner details, Z is a trailer. The B rows are the actual sales, so those are the ones I’ve kept. Once unified it’s ~7.4 million sales (as of July 2026).

Data quality

Each of the records began life as a ‘Notice of Sale’ form filled in by the purchaser’s solicitor or conveyancer and lodged with the land registry. For most of the dataset that meant a paper form (electronic lodgement only became mandatory in stages over 2019–2021). Decades of transcribed conveyancing paperwork is likely to need some cleaning. I’ve already noticed the published file-format documentation doesn’t always match the data. Two examples I hit right away:

• The historic-format spec says dates are stored CCYYMMDD, but they’re DD/MM/YYYY
• The same spec lists 20 fields per sale record. But the files have 22: every row carries an undocumented trailing ;;

These were both easy to handle. Looking through the data, I’ve noticed other issues that will take more fixing. Like postcodes that don’t match their suburb, prices of $1, a sale dated to the year 15, the same transaction lodged a hundred times.

Now I’ve got the data I’ll move on to auditing it, documenting issues, deciding what to fix and what to flag. The next few posts will probably be about that work.

A quick look before the cleaning

Before pulling the data apart, here’s one view of three suburbs over time: Liverpool in the south-west, Ryde in the north, and Maroubra in the east. Each row is a year, each distribution captures the spread of sale prices in that year. Note 2026 is only a half-year, as this was made in July 2026. You can click a suburb in the legend to toggle its visibility.

library(plotly)
library(dplyr)
library(scales)

dat <- readRDS("ridge_data.rds")
ridges <- dat$ridges; medians <- dat$medians
cols <- dat$cols; ord <- dat$order

SCALE_V <- 4.5
ymax <- 2026
pos <- function(y) ymax - y                       # 1990 at the top
gmax <- max(ridges$dens)
rgba <- function(hex, a) {
    r <- col2rgb(hex); sprintf("rgba(%d,%d,%d,%s)", r[1], r[2], r[3], a)
}

p <- plot_ly()
shown <- setNames(rep(FALSE, length(ord)), ord)
for (yr in sort(unique(ridges$year))) {
    for (sb in ord) {
        r <- ridges %>% filter(suburb == sb, year == yr) %>% arrange(x)
        if (nrow(r) == 0) next
        base <- pos(yr)
        p <- add_trace(p, type = "scatter", mode = "lines",
                       x = c(r$x, rev(r$x)),
                       y = c(base + r$dens / gmax * SCALE_V, rep(base, nrow(r))),
                       fill = "toself", fillcolor = rgba(cols[[sb]], 0.4),
                       line = list(color = cols[[sb]], width = 0.6),
                       legendgroup = sb, name = sb,
                       showlegend = !shown[[sb]], hoverinfo = "skip")
        shown[[sb]] <- TRUE
    }
}
for (sb in ord) {
    m <- medians %>% filter(suburb == sb) %>% arrange(year)
    p <- add_trace(p, type = "scatter", mode = "lines+markers",
                   x = m$median, y = pos(m$year),
                   line = list(color = cols[[sb]], width = 1.2),
                   marker = list(color = "white", size = 4,
                                 line = list(color = cols[[sb]], width = 1)),
                   legendgroup = sb, showlegend = FALSE,
                   text = paste0(sb, "<br>", m$year, "<br>median ",
                                 dollar(m$median, scale_cut = cut_short_scale()),
                                 "<br>", comma(m$n), " sales"),
                   hovertemplate = "%{text}<extra></extra>")
}
yrs <- sort(unique(ridges$year))
layout(p,
    title = list(
        text = paste0("<b>Three Sydney suburbs, 1990 to 2026</b><br>",
                      "<sup>Yearly sale-price distribution; the solid line ",
                      "connects each year's median</sup>"),
        x = 0, xanchor = "left", font = list(size = 16)),
    xaxis = list(title = "Sale price", range = c(0, 3.2e6),
                 tickvals = c(0, 1e6, 2e6, 3e6),
                 ticktext = c("$0", "$1M", "$2M", "$3M"), zeroline = FALSE,
                 fixedrange = TRUE),
    yaxis = list(title = "", tickvals = pos(yrs), ticktext = yrs,
                 showgrid = FALSE, zeroline = FALSE, fixedrange = TRUE),
    dragmode = FALSE,
    legend = list(groupclick = "togglegroup",
                  x = 0.99, xanchor = "right", y = 0.98, yanchor = "top",
                  bgcolor = "rgba(255,255,255,0.6)"),
    hovermode = "closest", margin = list(l = 45, t = 70)) |>
    plotly::config(displayModeBar = FALSE, responsive = TRUE,
                   scrollZoom = FALSE, doubleClick = FALSE)