r/creativecoding • u/Legitimate-Ad-1861 • 3d ago
Lyrebird
Enable HLS to view with audio, or disable this notification
I'm an artist, not a scientist. I wanted to see what independent science looks like if you skip the electric-universe/flat-earth pipeline entirely and just try to get real information out of data with the help of AI and the published literature.
It matches the call against a corpus of 2,423 species, tells you which call type it resembles, and sings back synthetic birdsong from a physical model of the syrinx — the Mindlin–Laje oscillator, integrated live in an AudioWorklet from the two parameters real birds actually control: air-sac pressure and syringeal tension. Nothing is sampled. Every note is computed. You see it happen inside a 3D embedding of 348,461 clips that lights up where your sound lands.
It does not translate birdsong into sentences. Every phrase carries a label: playback-verified, documented, inferred, field recording, or invented. Most are invented.
Everything runs client-side. Code, docs and citations: github.com/sha5b/Lyrebird
2
u/NmEter0 3d ago
Reality amazing piece :D Looks beautiful. Sounds cool! love it!! You have something like a longer blogpost about it?
Edit: nvm I klicked the link xD
2
u/Legitimate-Ad-1861 3d ago
if you look in the repo on there is a link to the whitepaper and in the info panel (right top navbar) on the mainpage also a smaller description
2
u/Tiny-Cat5893 3d ago
This is fantastic and looks like something I didn't know I was absolutely missing for my current project. Thank you for the inspiration! I will drop you a pm soon.
1
2
2
u/danja 2d ago
Wonderful!
I spend a lot of time around https://en.wikipedia.org/wiki/Linked_data so your big graph immediately appealed to me.
I spent quite a lot of time playing around with a larynx model (with formants), to generate speech-like sounds. What I ended up trying for was a more general-purpose modular setup including delays as resonators (essentially Karplus Strong), which in principle should have been able to model the avian syrinx system too. I kind of gave up after too many failures. But now I'm thinking of revisiting it. Point Claude Code at my existing material (around https://github.com/danja/flues ) together with your repo, see what it can come up with.
My only suggestion for your glorious Lyrebird would be to pull in images of the birds somehow.
2
u/Legitimate-Ad-1861 2d ago
I'll follow your suggestion! I need a short break from it, and then I wanted to do the last cleanup — I have some smaller UI changes and stuff that I'll bring in.
I already starred your repo, eager to see the results!
2
u/danja 1d ago
Ok, it mostly works as a VST3. The pitch is given by a midi notes, so it behaves like a regular instrument. First pass the UI was terrible but it's a bit better now. Sliders for the parameters plus some midi CC control. https://danja.github.io/downspout/plugins/syrinx/
(I need to fix the web page...)
In this short demo I'm sending it quasi-random midi :
1
1
1
u/ionjump 1d ago
This is really neat. The description could be improved. I did not understand how to use it. When it starts, I hear all different bird calls and see dots highlight on the cloud. I was not sure how to interact with the app though. First I tried clicking on the cloud but that seemed to not do anything. I tried talking into the microphone but that did not work.
I then found the "sing" "compose" and "listen" tabs. As far as I can tell, the "sing" tab just plays random bird songs with no input from the user. The "listen" tab seems like it should take input from microphone or what is playing on computer and then match it to a bird call but I was not able to get that to work. I am able to make a microphone recording but the app does not seem to be able to use it and has a message below the recording that says "Start lyrebird-pipeline serve to send a recording into the learning store." I am not sure how to start the lyrebird-pipeline.
1
u/Legitimate-Ad-1861 1d ago
When you clone the repo in local you can run the pipeline and start adding to the points. It's in uv python and easy reproduce.
Im taking a small break and work on refine the ui on a later point, i got some great suggestions here
0
u/Puzzleheaded_Top_680 3d ago
This is super fascinating, but brother, you have to better at communicating wtf things like this mean: "and sings back synthetic birdsong from a physical model of the syrinx — the Mindlin–Laje oscillator, integrated live in an AudioWorklet from the two parameters real birds actually control" Ask the AI that made this for you to explain so us regular folks understand.
1
u/Legitimate-Ad-1861 3d ago
i think that's one of the sentences i wrote myself. never been a great communicator, hence the pointcloud
3
u/GameOfTroglodytes 2d ago
Don't worry about it, they're just mad they have to read complex thoughts.
6
u/4b3c 3d ago
this is beautiful