# shivam garg Canonical website: https://shivamgarg.me/ This is the complete agent-readable version of Shivam Garg's portfolio. The main website has three sections: about, real work, and library. ## about I'm a cs student at the University of Maryland who got really into machine learning after taking an intro AI course. Now I spend a lot of my time trying to make neural networks do interesting things. While i'm an avid user of llms, no text on this site is computer-generated. Right now i'm exploring ways to push both multimodal ML and decision intelligence forward, with a growing focus on understanding human behavior via simulation. I'm a firm believer in doing. I also believe in exercising every day, in over-stressing the details, in not working in your bedroom, that life should be heavily spontaneous, that good taste requires a lot of exposure, that balance is overrated, that failure is never fatal, that your confidence changes everything, and that life needs a little whimsy to it. Whether youre interested in building something or want to share ideas, feel free to reach out to me on X. ## links - GitHub: https://github.com/shivamg05 - X: https://x.com/shivamxgarg - LinkedIn: https://www.linkedin.com/in/shivam-garg-0520a5231/ - Hugging Face: https://huggingface.co/shivamg05 - Email: sgarg23@terpmail.umd.edu ## experience ### amazon - Role: swe intern - Period: summer 2026 - Description: built control-plane automation that dynamically rebalances traffic across the routing sards of amazon's service mesh load balancer, safely relocating services and optimally placing new ones. - Takeaway: second order effects compound quickly ### amazon - Role: swe intern - Period: summer 2025 - Description: built end-to-end S3 dataset support into amazon’s mission-critical query-as-a-service Datapath platform. lots of async java, aws service integrations, and late-night load testing. - Takeaway: learned that production systems at scale are 80% failure mode handling, 20% happy path development ### drymint - Role: cofounder - Period: summer 2019 - present - Website: https://drymint.com - Description: created and scaled a wellness brand from the ground up. what started as selling candles in a children's business fair turned into years of product iteration, taste refinement, and learning what it takes to sell. - Takeaway: if you can't sell effectively, having a good product means nothing ### sgsoles - Role: founder - Period: summer 2020 - summer 2022 - Instagram: https://www.instagram.com/sg.soles/ - Description: customized and resold sneakers. made relationships with brick and mortar store owners that im still friends with today. - Takeaway: this is where i realized that knowing people expedites processes ten fold ## projects ### alfred - Project page: https://shivamgarg.me/alfred/ - Category: agents - Year: 2026 - Summary: iMessage-native AI journal that actually remembers - Demo video: https://shivamgarg.me/videos/alfred-demo.mp4 - Stack: typescript, sqlite, chromadb, imessage, knowledge graphs #### overview alfred is the friend who catches the thought before it disappears, remembers it without making a big deal about it, and brings it back when it actually matters. you text it like a person, and it builds a structured understanding of your life over time by extracting atomic facts, connecting to external apps (e.g. Todoist, Calendar), organizing them into a knowledge graph, and proactively surfacing things when theyre relevant. #### technical details messages get decomposed into immutable fact nodes across three levels: L0 (ephemeral events), L1 (behavioral patterns), L2 (identity/values). facts are never edited, corrections create new nodes with directed edges, so the graph is versioned by default. retrieval merges chromadb semantic search with keyword search via reciprocal rank fusion, then expands through the graph. expired L0 facts get consolidated into L1 patterns over time, and supported L1s promote to L2 weekly. the proactive engine hooks into this structure to surface pattern observations, external synthesis, and check-ins, all gated by an LLM judge. #### what i learned memory is not the same thing as storage. dumping everything into embeddings is easy, but it breaks fast when the inputs are tiny, casual, and constantly changing. memory is a living system: facts decay, patterns compound, and history needs to stay traceable ### smollm2 reimplementation - Project page: https://shivamgarg.me/smollm2-reimplementation/ - Repository: https://github.com/shivamg05/smollm2-dupe - Category: LLMs - Year: 2026 - Summary: A from-scratch PyTorch implementation of SmolLM2-135M - Stack: pytorch #### overview i wanted to internalize how design decisions are made when creating language models, and i felt like the best way to build that knowledge was to reimplemnt a foundational model from scratch without the help of ai-generated code. smollm2 felt like a good target because it's relatviely compact but still uses a modern approach. #### technical details implemented a decoder-only transformer architecture from scratch in pytorch, including grouped-query attention, rope positional embeddings, and swiglu feed-forward layers. wrote a custom training loop with micro-batching, gradient accumulation, and a linear warmup + decay scheduler to match the original setup. also implemented kv caching for efficient autoregressive inference, reducing redundant attention computation during generation #### what i learned reimplementing a model from scratch forces you to resolve the gap between the high-level architecture and low-level details you don't read about. ### neural decoder - Project page: https://shivamgarg.me/neural-decoder/ - Repository: https://github.com/shivamg05/brainstorm - Category: brain-computer interface - Year: 2026 - Summary: real-time neural decoder built for edge devices - Stack: pytorch, ml systems, signal processing, bci #### overview for a bci hackathon, i built a causal decoder that takes in raw ECoG signals from cortical electrodes and predicts the associated frequency heard in real time. the main challenge was balancing accuracy with latency and model size so the system could live inside something like a closed-loop auditory prosthetic or neural interface. super cool use case imo, and ended up placing second for this track in the hackathon which was a plus! #### technical details i experimented with several architectures (eegnet, eeg-tcnet, and a custom cnn+gru hybrid). the final model (cnn+gru) extracts compact spatial features with a small cnn, then models temporal dynamics with a gru for fully causal predictions. preprocessing played a huge role in accuracy and included bandpass filtering with envelope extraction, emphasizing high-gamma activity, and mapping the 1024 electrodes onto a 2d grid. balancing the previously skewed sample distribution during training and quantizing the model post-training also helped increase accuracy and decrease the model size. #### what i learned signal processing matters as much as the model, and bci is less mysterious than most people assume. ### x-mcp - Project page: https://shivamgarg.me/x-mcp/ - Repository: https://github.com/shivamg05/x-mcp - Category: infra - Year: 2026 - Summary: mcp server to interact with the x (twitter) api - Stack: python, FastMCP, OAuth 2.1, FastAPI, X API #### overview x-mcp is a local mcp server that allows mcp clients to interact with the x (twitter) api. it provides tools for searching posts, retrieving user information, and creating posts, while isolating oauth, tokens, and credentials on the user’s machine. #### technical details implemented a stdio-based mcp server with a local oauth 2.1 + pkce flow using a lightweight fastapi callback server for authorization. each user connects their own x developer app, with access tokens stored and refreshed locally. the server exposes tools (search, read, create post) with explicit schemas, allowing llm clients to safely invoke real-world actions without handling secrets or credentials directly. #### what i learned learned that a good portion of the complexity in tool integration lives outside the tool itself. auth lifecycles and token storage are just as important as api surface area. ### Audio-SmolVLA - Project page: https://shivamgarg.me/audio-smolvla/ - Paper: https://shivamgarg.me/audio-smolvla/paper/ - PDF: https://shivamgarg.me/papers/Audio-SmolVLA-paper.pdf - Demo video: https://shivamgarg.me/videos/Audio-VLA-demo.mp4 - Category: robotics - Year: 2025-2026 - Summary: a lightweight VLA extended with real-time audio perception - Stack: vlas, audio signal processing, multimodal ml, pytorch, robotics #### overview audio-smolvla is an attempt to extend a small vision-language-action model with audio perception, so a robot could respond to sounds in addition to visual and language context. audio is an important signal for real-world interaction in occuleded settings or in settings where reaction to alarms/noises is necessary. we built around smolvla and explored whether audio could be fused in without destroying the model’s pretrained visuomotor capabilities, while still improving performance on tasks where sound actually matters. If interested in reading our paper, please reach out! #### technical details we designed an audio fusion pipeline on top of smolvla, using a pretrained audio encoder and a small mlp projector to map audio embeddings into the smollm2 language embedding space. training was split into two stages: first, aligning audio and text representations with a contrastive + generative objective on audioset, then freezing that stack and finetuning the action expert on a custom robotic manipulation task. we also extended the lerobot pipeline itself to support audio collection and training, and collected a custom dataset with the so-101 arm where the robot had to react to a bell cue during a pick-and-place task. #### what i learned adding a new modality is not just a modeling problem, it's also a systems problem since data pipelines, preprocessing, and training throughput became just as limiting as the policy itself. ### groundhog - Project page: https://shivamgarg.me/groundhog/ - Repository: https://github.com/shivamg05/groundhog - Demo video: https://shivamgarg.me/videos/groundhog-demo.mp4 - Category: agents - Year: 2025 - Summary: autonomous web agent built without an agent-building framework - Stack: pytorch, Qwen-VL, LoRA, python, selenium #### overview most web agents rely on massive closed-source models and abstraction frameworks like langchain. i wanted to strip that away to understand the raw mechanics of the entire stack, so i built a locally-run agent that operates as a closed control loop: it perceives the browser state through both distilled dom trees and screenshots, maps high-level user goals to low-level selenium actions, and self-corrects based on the resulting state changes. #### technical details finetuned qwen2.5-vl-7b-instruct on a processed subset of mind2web using qlora (4-bit) adapters. the core prompt strategy interleaves simplified dom representations with viewport screenshots to ground the model's spatial reasoning. the inference loop runs locally, parsing natural language thoughts into structured webdriver calls (click, type, scroll) and handling error recovery when elements drift. #### what i learned learned that context density matters more than context length. raw html is 90% noise to an llm, so passing the dom in as is causes signal loss. by stripping non-interactive nodes while preserving semantic anchors (headers, labels), i reduced token overhead by ~70% without losing spatial reasoning. ### socialmuse - Project page: https://shivamgarg.me/socialmuse/ - Website: https://socialmuse.dev - Demo video: https://shivamgarg.me/videos/socialmuse-demo.mp4 - Category: web app - Year: 2025 - Summary: platform that helps brands discover emerging social media trends that align with their unique voice - Stack: python, postgres, FastAPI, Vite, TypeScript, Heroku, OpenAI API #### overview as a cofounder of [drymint](https://drymint.com), i realized that for business owners who aren't chronically rotting online, organic social media growth is hard. so to help them find their footing and use validated inspiration, i decided to make a platform to find hot/upcoming trends that are relevant to each user's brand. #### technical details implemented a multi-stage pipeline that scrapes tiktok data (transcript, comments, song transcript, etc), embeds video text and audio signals, and clusters content into emergent trend types. trends are matched to brands using bi-encoder similarity, then reranked with an llm-as-a-judge step that evaluates voice alignment and adaptability. #### what i learned learned that good product abstractions hide complexity without hiding intent. there are many steps done under the hood in order to minimize user effort but maximize usage. ## library - https://shivamgarg.me/images/library/01.jpeg - https://shivamgarg.me/images/library/02.jpeg - https://shivamgarg.me/images/library/03.jpeg - https://shivamgarg.me/images/library/07.jpeg - https://shivamgarg.me/images/library/08.jpeg - https://shivamgarg.me/images/library/09.jpeg - https://shivamgarg.me/images/library/10.jpeg - https://shivamgarg.me/images/library/11.jpeg - https://shivamgarg.me/images/library/12.jpeg - https://shivamgarg.me/images/library/13.jpeg - https://shivamgarg.me/images/library/15.jpeg - https://shivamgarg.me/images/library/16.jpeg - https://shivamgarg.me/images/library/17.jpeg - https://shivamgarg.me/images/library/18.jpeg