Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Rainfly - An AudioWorklet DSP Playground [MVP] #395

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
Open

Rainfly - An AudioWorklet DSP Playground [MVP] #395

wants to merge 13 commits into from

Conversation

terryzfeng
Copy link
Collaborator

@terryzfeng terryzfeng commented Aug 25, 2024 •
edited
Loading

This PR is to introduce Rainfly: An AudioWorklet DSP Playground for Web Audio. This online AudioWorklet IDE streamlines writing custom JavaScript AudioWorklet code, building Web Audio DSP graphs, and recording output for easy visualization, analysis and validation.

Demo: https://googlechromelabs.github.io/web-audio-samples/rainfly/

Rainfly MVP Features:

  • Monaco Code Editor for writing AudioWorkletProcessor and AudioContext graph
  • Realtime DSP execution via AudioContext, can pause and play
  • Recording and visualization of DSP output a la Canopy
  • Example AudioWorklet code and boilerplate
  • Error console for JS execution error feedback
  • Exporting JS code or recorded output to a .wav
  • Vim support
  • Built with Svelte

Authors:

GSoC 2024

@terryzfeng terryzfeng added the gsoc24 Google Summer of Code 2024 label Aug 25, 2024
@terryzfeng terryzfeng linked an issue Aug 25, 2024 that may be closed by this pull request
AudioWorklet DSP Playground #389
Open
12 tasks
@terryzfeng
Copy link
Collaborator Author

@hoch @mjwilson-google I've created the Rainfly directory just directly under src. Just wanted to confirm directory location or if it should be under a specific directory? It looks like eleventy builds all current directories.

@terryzfeng terryzfeng marked this pull request as draft August 25, 2024 06:49
hoch
hoch reviewed Aug 27, 2024
View reviewed changes
Copy link
Member

@hoch hoch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The directory location looks good to me.

But don't we need to exclude this directory from Eleventy's scanning? Can we include that change in this PR as well?

@terryzfeng
Copy link
Collaborator Author

That sounds good. Eleventy doesn't build this directory yet but I'll make sure that it's all working. I'll copy the rainfly repo into this directory for this PR.

@hoch hoch mentioned this pull request Sep 3, 2024
Open
@Kizjkre Kizjkre marked this pull request as ready for review October 1, 2024 18:54
@Kizjkre Kizjkre requested a review from hoch October 4, 2024 05:50
hoch
hoch approved these changes Oct 4, 2024
View reviewed changes
Copy link
Member

@hoch hoch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with nits

It's okay to land this as is, but I am asking to work on nits from the review as a follow-up.

.eleventyignore Outdated Show resolved Hide resolved
.github/workflows/main.yml
@@ -19,7 +19,7 @@ jobs:
- name: Install Node.js
uses: actions/setup-node@v3
with:
node-version: 16
node-version: 18
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious - is 18 required?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, 18 is required for Svelte.

src/_data/audioworklet_data.yaml Outdated Show resolved Hide resolved
src/rainfly/src/lib/components/ActionButton.svelte Outdated Show resolved Hide resolved
src/rainfly/src/lib/components/Editor.svelte Outdated Show resolved Hide resolved
src/rainfly/src/lib/components/nav/items/NavExamples.svelte Show resolved Hide resolved
src/rainfly/src/lib/components/nav/items/NavFile.svelte Outdated Show resolved Hide resolved
src/rainfly/src/lib/utils/audio-buffer-to-wav.js Outdated Show resolved Hide resolved
src/rainfly/src/lib/utils/audio-buffer-to-wav.js Outdated Show resolved Hide resolved
src/rainfly/src/lib/utils/file-utils.js
* an object with the name and data of the file
*/
export async function fetchTextFile(url) {
let filename = url.split('/').pop();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see any error handling here. Is jszip handling errors somehow?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hoch Do you mean error handling for the filename handling or the fetch requests (next few lines)? jszip isn't involved in this part of the code. I've gone ahead and wrapped the fetch requests in a try catch.

mjwilson-google
mjwilson-google reviewed Oct 17, 2024
View reviewed changes
Copy link
Collaborator

@mjwilson-google mjwilson-google left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for continuing to work on this!

I didn't try running it yet, but I have some small comments. Since Hongchan has already approved, please feel free to resolve my comments and land if necessary.

src/_data/audioworklet_data.yaml Outdated Show resolved Hide resolved
src/rainfly/src/lib/actions/portal.js Show resolved Hide resolved
src/rainfly/static/examples/bypass/main.js
@@ -0,0 +1,9 @@
// Use `context` as the AudioContext
// @sampleRate = 48000
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a parameter? Generally the output device sample rate is used if nothing is specified (https://webaudio.github.io/web-audio-api/#dictionary-audiocontextoptions-members).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes this is a parameter that we preprocess. If not specified, default is also 48000.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would the Canopy parameter syntax of @sampleRate 48000 be preferred?

src/rainfly/static/examples/sine/main.js
@@ -0,0 +1,7 @@
// Use `context` as the AudioContext
// @sampleRate = 48000
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as for the other sampleRate location.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mjwilson-google mjwilson-google mjwilson-google left review comments

@hoch hoch hoch approved these changes

@Kizjkre Kizjkre Awaiting requested review from Kizjkre

Assignees

@Kizjkre Kizjkre

@terryzfeng terryzfeng

Labels
gsoc24 Google Summer of Code 2024
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

AudioWorklet DSP Playground
4 participants
@terryzfeng @hoch @Kizjkre @mjwilson-google