<aside>

</aside>

<aside> 📍

You are not lost, you are here.

You have arrived at a sub page in the AIxDESIGN Archive.

This page is part of the Open Culture Tech project.

</aside>

<aside>

INDEX

</aside>

0_ About Saz-e-yaad

Saz-e-yaad came out of an Open Culture Tech Artist Residency with Lacuna, 𝔥𝔢𝔵𝔬𝔯𝔠𝔦𝔰𝔪𝔬𝔰, and AIxDESIGN. Over the course of 1 year, we explored how we might develop audio generative tools based on melodies, rhythms and composition of Kashmiri folk music. We investigated where we could source archives of Kashmiri folk music, trained 22 audio generation models, and developed a Standalone App (beta). We learned a ton along the way. This is our attempt at documenting what we learned and the questions that were left unaswered (future research brief for us or someone else?)

<aside> 🎁

1_ What we made

Over the residency we shipped one instrument and three generative models, with an ML-ready dataset to follow. Everything below is generated from / runs on Kashmiri folk recordings sourced from Ravimech Studios. The software can be run on consumer grade laptops. We tested on Macbook Pro M4 Max.

• VST Standalone App // Saz-e-Yaad : about + link

  • About:

    Saaz-ee-yad is a desktop instrument that wraps our trained Kashmiri-folk generative models behind a single GUI. You pick a model (Full/Melody, Harmony, Rhythm, Vocals), choose how to steer it; unconditional, an audio prompt you drag in, or a text prompt and it generates new audio in the style of the dataset. While the first sample renders, decoded chunks are crossfaded into a live streaming window so you hear the piece forming instead of waiting on a blank screen. Generated takes land in a sample panel; "Continue Selected" feeds any previous output back in as a prefix so you can chain a track forward. Controls expose the real knobs of the sampler: length, diffusion steps, temperature, seed scale ("surprise"), number of samples, and an optional seed for repeatable renders. Output is plain .wav you drag straight into Ableton Live or any DAW.

    • VST is a software format that allows digital instruments and audio effects to be used inside music recording programs (DAWs) like Ableton.

  • Under the hood: TorchScript model inference, music2latent for audio↔latent, CLAP (laion/larger_clap_music) for audio/text conditioning, MPS (Apple Silicon) with automatic CPU fallback.

  • Links: [TBD — release/download URL]`

Model 0 // Saz-e-yaad Ravimech

• About

  Our full-spectrum model — the one trained on complete, un-separated mixes (243 curated Ravimech tracks). It generates the most "whole" Kashmiri-folk-sounding output: melody, accompaniment and texture together. Architecture is a Continuous Autoregressive Model (CAM) operating over music2latent latents, conditioned through CLAP cross-attention; an autoregressive transformer backbone predicts continuous latent frames and a per-frame diffusion sampler renders them. Timbrally convincing over ~30s, with occasional long-range structural drift. The model is bundled with the software.

• Links [TBD — model card / weights URL]

Model 1 // Saz-e-yaad Melody

• About
• Links

Model 2 // Saz-e-yaad Harmony

• About

  A melody/harmony model trained on tracks with drums and percussion removed (with a commercial third party AI noise suppression and source separation), so it learns pitched, harmonic content without the rhythmic anchor. Trained on a small curated subset (~20 tracks). It produces dreamy, floating melodic phrases — excellent as pad/texture material, weaker on driving structure. Same CAM + music2latent + CLAP stack as Melody. The model is bundled with the software.

• Links

[TBD — model card / weights URL]

Model 3 // Saz-e-yaad Rhythm: Results + hypothesis for next steps

• About

  ?

• Results: The most structurally coherent of the three. Albeit showed artifactas from AI source separation and excessive silence. It was trained on isolated percussion. Kashmiri folk rhythm patterns are comparatively modeled, so the model locks onto them and holds groove better than Melody or Harmony hold their long-range structure. Both bundled in the App.

• Hypothesis / next steps

• Links

  [TBD — model card / weights URL]

Coming soon // ML-ready dataset

A cleaned, documented release of the preprocessed Kashmiri-folk dataset (music2latent latents + CLAP tokens) is in preparation. Pending rights clearance with Ravimech Studios, metadata enrichment, and higher-quality source audio where possible.

</aside>

2_ How it started

A note from Lacuna (the artist)

<aside>

I am an electronic music composer, educator and researcher based in Utrecht. Originally from India, I made the move to NL to pursue a masters in Music and Technology at HKU. My artistic practice centers around the reinterpretation of heritage sounds through contemporary electronic music, sound system culture, and emerging technologies. As a composer and researcher, I have been working extensively with archival materials, including sounds from the Netherlands Institute for Sound and Vision (Beeld & Geluid) to create live sets and experimental compositions.

These explorations have led me to an evolving research interest investigating how digital audio workstations (DAWs), hardware, and AI can be used to process, manipulate, and contextualize heritage audio. As an electronic music producer, my work explores Jamaican sound system culture with emphasis on dub rhythms and their modern evolution in sound design. Through my current work i'm bridging heritage sounds with sound system culture and attempting to create a unique blend of both worlds.

My current master's research at HKU focuses on how sound archivism can be creatively reimagined within music production.

I am particularly interested in how cultural memory, migration, and decolonial thinking intersect within sound-based art.

</aside>

• @mihir raina your application in case you want to reference it

Project Intention

Through the Open Culture Tech Residency, Lacuna wanted to deepen his engagement with AI-driven sound design.

Objectives before we started

• AI-Driven Heritage Sound Processing: Experiment with AI-powered tools to create new workflows for treating heritage recordings, investigating ways to maintain their cultural essence while transforming them into contemporary musical textures.

What we did

We did this! story

• Hybrid AI and DAW Integration: Develop methods for seamlessly integrating archival sounds into modern production environments, using AI as a bridge between traditional and electronic music forms.

We *sorta* did this We first tried with realtime audio generation (RAVE), but story

3_ What we did / The Process

‣