Writing software is my passion. Building web applications, websites, tools, utilities, or artsy code sketches is what I do in my free time and want to do for my career as well. Here is a collection of the notable software projects that I've undertaken and published. The list isn't comprehensive and is always growing.
An experiment for generating embeddings from an audio sample library and a UI for visually exploring the sample library in 2D space. The samples processed with MFCC for feature extraction, the embeddings are generated using UMAP, and the UI is built with Svelte + D3.The tool is integrated into the sample library module of my web synth project.
A combined spectrogram + oscilloscope module for my web-synth project. It integrates into web synth and can be used to analyze any signal. The visualizations are built using lots of cool modern web tech including SharedArrayBuffer, Atomics, OffscreenCanvas, Rust compiled to WebAssembly + Wasm SIMD, and more.
I added support for AVX-512 to the popular llama.cpp LLM project. This sped up inference by a bit on processors that support it. The work was quickly obsoleted by other optimizations to the rapidly evolving project, but it was a fun experience to contribute to that effort.
A small utility added to web synth to allow users to build their own wavetables by adding in harmonics individually. This builds on the existing wavetable support for the FM synth, allowing users to build their own wavetables in addition to importing them from files.
I picked active development of web synth back up with the goal of producing "real music" using the tool. I prioritized filling some gaps in features as well as polishing up the application as a whole. Some of what I worked on includes building an OTT-style multi-band compressor, a mixer UI with level visualization, revamping the MIDI editor to support multiple tracks, adding a welcome page, building a custom dynamic bandpass filter, moving MIDI scheduling to the audio thread, adding time-tracked automation tracks, and fixing tons of bugs.
A continuation of my 3D work with Three.JS. This scene was inspired by the "Rainy Apartments" level from the game Yume 2kki. Everything is modeled by hand in Blender and rendered with Three.JS, using lots of postprocessing for a distance blur effect. Integrates with web synth for rain sounds.
A collaboration with @N8Programs, porting his godrays shader for Three.JS into a library that integrates with the pmndrs postprocessing library. The shader is a high-quality volumetric lighting effect that generates very cool looking godrays in any Three.JS project, with lots of controls for configuring the effect.
While working on my Three.JS projects, I used the Spector.JS tool to debug performance issues and was encountering some crashes when trying to load large scenes with it. I contributed some fixes to expand support to much larger scenes to the project, and speed up the extension significantly in some cases.
Web-based utility to combine multiple seamlessly tiling textures into a single larger texture. This is a useful alternative to expensive runtime methods like hex tiling. It works very well when using AI-generated seamless textures, and I used it in some of my 3D sketches. Written in Rust + compiled to WebAssembly.
Continuation of my 3D work with Three.JS and Blender. An alien-looking environment with bright pink animated volumetric lighting. Includes my first proof-of-concept integration with my web synth project to play some dynamically generated music.
The result of a lot of experiments with Three.JS and Blender. Full first person controls with collision detection + physics, lots of custom shaders + textures, and web synth integration for sound effects. The scene itself consists of a bridge over a void with many floating monoliths and an alien-looking environment.
A collection of small interactive game-like 3D environments and demos. Somewhat inspired by games like Yume Nikki/Yume 2kki, Over the Void, and others. There is no overarching goal or theme, but this work may lead to a larger project in the future. The demos are all browser-based, rendered with Three.JS + WebGL and use 3D models I'm creating myself in Blender. Includes some work with procedural materials and custom shaders inspired by projects on Shadertoy.
While working on an idea for an interactive visualization of RNNs, I ended up going down a rabbit hole of exploration and experimentation in the realm of neural networks, machine learning, and information theory. Work included developing a new activation function, using ideas from other research papers to estimate the Kolmogorov and boolean complexities of different functions, and reverse-engineering a neural network's clever solution to binary addition. This work culminated in a lengthy blog post full of interactive visualizations to illustrate the concepts involved.
An anime recommendation engine powered by a custom-built neural network. Using a dataset of user profiles from the popular site MyAnimeList, the network was trained entirely in the browser via Tensorflow.JS. The whole thing is implemented as a SvelteKit app with predictions happening directly within the app itself using tfjs-node and a background thread. Additional post-processing is applied to work around imbalances due to large popularity differences. It also contains support for determining which ratings contribute most to each of the recommendations.
In a similar vein to the Music Galaxy (a few projects down), this is an embedding visualization that is constructed based off of relationships between anime. A co-occurrence matrix is constructed from a large dataset of user anime rankings which is then used to construct a variety of embeddings using different algorithms (PyMDE + GGVec) and parameters. The embedding is then projected into 2D with t-SNE. The visualization itself is built with PIXI.JS and has the ability to search + fly to anime as well as load in user profiles from MyAnimeList to visualize where one's own anime rankings lie in the broader context.
A browser-based tool for building, training, visualizing, and experimenting with neural networks. Written in Rust, compiled to WebAssembly with SIMD. It can learn arbitrary functions via randomly sampled examples, showing its progress on a 3D surface plot. There is support for changing all network parameters like the number of layers, the activation function for each layer, learning rate, and more. It also includes visualizations of individual neurons' responses to explore the internal workings of the network.
A small 3D world built using Three.js and Blender that runs in the web browser. Accompanied by a YouTube mini-series of short videos going over the process of creating it and explaining some of the background concepts.
Music Galaxy is is the culmination of several months of work with artist relationship data from Spotify. It is an interative 3D visualization of the relationships between over 70,000 artists, positioning them in such a way that artists with similar audiences are positioned closer together in space. It also supports connecting to users' Spotify accounts to visualize where their personal favorite artists exist in context of the greater musical universe.
`rnnoise` is an open source noise suppression library based on a recurrent neural network. It is used in various software, and I use it myself on my desktop as a part of `pulseeffects`. I located a hot function in the application used for actually applying the neural network's weights which was taking up significant portion of its CPU time, and I realized it could be sped up significantly with SIMD. I converted the code to use SIMD intrinsics which yielded a huge reduction in CPU usage. I also contributed a pull request to the rnnoise repository with the changes.
A small library adding support for response compression to the Rocket web framework for Rust. It uses the `async-compression` library to provide support for gzip, deflate, brotli, and zstd compression.
An interactive web-based visualization of related artists on Spotify based on data from the Spotify web API. It allows patterns in music styles and artists to be seen in a way that moves beyond traditional genre classifications. The visualization is integrated into my Spotifytrack project.
A fully in-browser MIDI editor built as a part of my broader browser-based audio synthesis platform (see below). It's the second browser-based MIDI editor I've made, and fixes many of the issues from the first one. It's currently only available as a module of the web synth platform, but I plan to make it into a standalone webapp as well.
As I've delved deeper into audio synthesis and DSP programming topics, I've started recording my findings and creating a personal knowledge repository using Foam, a personal knowledge management and sharing system for VSCode (https://github.com/foambubble/foam). Topics range from FM synthesis to speech/singing synthesis as well as some documentation for my web synth project. I plan to continue expanding this knowledge base and possibly extend it into other areas that I spend time learning about in the future.
An 8-operator frequency modulation synthesizer built in Rust and WebAssembly with SIMD that runs 100% in the web browser via WebAudio. Built as a standalone demo, it is also integrated into the full web-based audio synthesis and experimentation platform I'm building.
Built in a weekend, this website does the same thing as my previous project osu!track (which took me months to build). It allows players of the rhythm game Quaver to track their progress and view how their stats progress over time as well as providing visualizations of their hiscore distribution and a few other features.
A little toy game I'm building for fun based off an old idea from years ago. It's a great opportunity to try out WebAssembly in the frontend, do some linear algebra and graphics programming, and do something creative. It's mostly a fancy tech demo right now and may never be anything more, but it's a ton of fun to work on.
I built and managed all of the tech for the revival of online concert series SPF420. This consisted of a fully custom streaming setup through which multiple artists could stream to my server and be displayed side-by-side on an output livestream and a website featuring a fully custom-built chat with moderation and interactive cursor trails + stickers. The first event on March 28 topped out at over 250 concurrent viewers and lasted nearly 6 hours. More similar events are planned.
I helped out a team of people organizing a set of three simulatneous concerts in San Francicso, New York, and Chicago connected by live stream and streaming online. I built the website (https://syncup.world) and handled the technical side of the livestream and web experience. The website was fully responsive with advanced styling to maintain the look of a poster on devices of all sizes and aspect ratios. However, all of the concerts were canceled due to COVID-19.
A Work-In-Progress site showing users' top songs and artists and listening patterns over time, providing a shareable link that they can use to show it off to friends. May include features for tracking the popularity or artists/songs/etc. over time as well.
A fully-featured TypeScript library defining a self-contained 100% type-safe Redux architecture. Jantix staticly ensures the validity and cohesion of your actions/reducers/state and provides a very clean, 100% strictly typed interface through which to interact with them.
An interactive ant colony simulation that I created as a project for my Simulation and Modeling class at school. Made using my Minutiae simulation framework, ants lay pheromones and work together to gather food in a world full of obstacles.
A homepage for myself with the primary purpose of showcasing my professional skills and achievements as well as personal projects and interests. I aimed to create something that demonstrates my skills as a web developer as well as gives others insight into myself and what I do.
The goal of this project is to create a web-based music creation application that can be used entirely within the browser. Since the modern WebAudio API provides everything necessary to create complicated synthesizers and perform advanced audio processing, I believed that the web could work as an accessible environment in which to build this.
A React port of the control-panel (https://github.com/freeman-lab/control-panel) library. I maintained 100% visual appearance and functionality parity with the original library while making it usable from React. I also added several additional features such as making panels draggable, making UI state external, and adding an Proxy-based interface for manipulating the UI state externally.
Server and TUI for managing and deploying static websites and files. Deployments are created as subdomains, allowing a directory of static files (such as one produced by `yarn build`) to go from your computer to being hosted at `project.ameo.design` with a single command.
A friend and I wanted to work on a game together, trying out some tech that we were interested in and building up all parts of a game from scratch. I handled the frontend and the physics while my friend focused on setting up the backend server. At the end, we came up with a networked physics engine based on a popular Rust physics framework that runs the same code on both the client and the server. A variety of techniques were used for handling network delay, synchronizing state, and keeping track of game data both internally on the client as well as on the server.
A tool for viewing data about the popularity of various stocks on the Robinhood brokerage. Allows users to view the most (and least) popular stocks held by its users and view trends over time. Robintrack gained significant popularity, with several people reaching out to use the data it collected for school or personal research and the Robinhood team reaching out to me directly about it.
A small demo of volumetric rendering via raymarching in the web browser using Rust, Emscripten/asm.js, GPU.js, and Emscripten. It creates a matrix of 3D noise and then pans around it, rendering the results to a canvas. All of the raymarching algorithmic and linear algebra was implemented from scratch, and the idea for the project was for it to serve as an excercise forme to learn how the whole process works from start to finish.
A web application developed using Rust + WebAssembly for creating and visualizing compositions of noise functions. 3D noise functions are projected onto a 2D canvas after being mapped through a colorizer function, creating intricate and psychedelic effects. Includes a web backend for sharing + browsing compositions from other users.
A simulation engine that operates on a finite 2-dimensional universe populated by cells and entities. It is designed to provide a modular system on top of which simulations of various types can be designed. Can be compiled into WebAssembly and run in the browser, optionally including client/server functionality. Minutiae is used in several of my other projects.
A live depth-of-market (DOM) web-based visualization for live orderbook data from the Poloniex cryptocurrency exchange. Allows traders and quantitative analysts to gain unique insight into the current state of the market by viewing complex market activity in real time.
A web application providing an overview of a user's portfolio on the Poloniex cryptocurrency exchange. It shows detailed visualizations of current holdings, demonstrates how portfolio value has changed over time, and provides a variety of advanced analytics detailing a user's trading history and account activity.
TickGrinder is a high performance algorithmic trading platform written primarily in Rust. It is designed with the goal of efficiently processing event-based market data as quickly as possible in order to automatically place and manage trades.
Site that tracked the gameplay statistics of League of Legends players. Users could view as their rank changed as they played and view a variety of other statistics about their gameplay habits. Elotrack was eventually taken offline due to the League of Legends API that it made use of being discontinued.
Site where players of the game osu! can track their progress and view the progression of others. Pulls data from the osu! API, storing time series data and creating a variety of charts, plots, statistics, and other visualizations for a variety of different metrics. An in-game chat bot and Discord bot were also written to allow users to update their stats from in-game. Thousands of players a month continue to use osu!track.