Skip to content
ofurkan.coWP No. 26-01
Oldenburg --:--
← §1 Work · Entry [8]

Discogs Data Machine

Tool2026Solo, end to end
Figure A.8. Illustrative figure, generated from the project name.
Abstract

A family of tools that turns the monthly Discogs music database dumps (tens of GB of XML) into clean CSVs and publishes them to Kaggle, fully automated.

Stack
Python · Swift · Kaggle API · Streaming XML · Homebrew
Open live ↗Replication files on GitHub ↗

Where music and data meet. Discogs publishes its entire catalogue every month as gzipped XML; the releases dump alone is around 10 GB compressed. I wanted it as tidy CSVs, so I built the tools, then automated the whole thing.

The family

  • DiscogsGUI: a desktop app with multi-threaded downloads, smart extraction, XML to CSV conversion and a cover-art generator. A native Swift/SwiftUI rewrite is in progress.
  • DiscogsCLI and a Homebrew tap for the terminal crowd.
  • Discogs Kaggle Sync: the monthly robot.

The robot

Every month it checks for a new dump, downloads with resume-on-reconnect, converts straight off the gzip stream (deleting each archive as soon as its CSV exists to keep peak disk usage low), generates a cover image, publishes a new Kaggle dataset with column descriptions, fills in the metadata Kaggle’s public API silently drops, and publishes a starter notebook. It only cleans up once the dataset’s usability score is a perfect 1.0.