gelu

Vision Transformers Needs Registers. And Gated MLPs. And +20M params. Tiny modality gap ensues!

Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models

Implemented GPT from scratch

Optimized CUDA kernels for NVIDIA GB10 Blackwell (sm_121, DGX Spark). RMSNorm + GELU. First sm_121 kernel on HuggingFace Kernel Hub.

Vision Transformers Needs Registers. And Gated MLPs. And +20M params. Tiny modality gap ensues!

"The 'Activation Functions' project repository contains implementations of various activation functions commonly used in neural networks. "

A GPT Model To Generate Text

C++17 ONNX inference optimizer + CPU runtime for Apple Silicon. Operator fusion via IR passes, Accelerate AMX-backed sgemm; benched vs ONNX Runtime CPU EP on DistilBERT (1.26x baseline speedup, 6.99x ORT on raw MatMul).

Accelerated AVX2 math kernels with certified SPEAR polynomial approximations (gelu, erf, tanh) - x15-34 vs libm

Annotated vanilla implementation in PyTorch of the Transformer model introduced in 'Attention Is All You Need'.

Here, we will provide a PyTorch regime to handle the partial differential equation solution of the heat equation by executing Deep Kolmogorov Method of Beck et. al.

Backward pass CUDA kernels for fused GEMM+Bias+GeLU on SM75 (Turing). Float4 vectorized WMMA kernels validated against PyTorch autograd. 3.1x faster than autograd at M=1024. 24/24 tests passing.

PyTorch implementation of normalization-free LLMs investigating entropic behavior to find desirable activation functions

Independent research on inactive-neuron entry-exit dynamics, with mathematical analysis, archived experiments, reproducibility guides, and a connected research roadmap.

This repository contains the code and the report for the coursework of INFR11031 Advanced Vision, a postgraduate course offered at The University of Edinburgh. The task was to train on limited and improve the accuracy of the ResNet-50 classifier on a small subset of the ImageNet dataset containing 50K training images and 50K test images. Achieve…

exploration of modified parametric GeLU (\lambda-Gelu)

This folder contains the supporting code and data for arXiv:2607.03664

Microphone Array-Based Direction of Arrival of Gunshot Detection .Gun violence remains a critical concern. Identifying the precise location of a gunshot—or getting as close as humanly possible—is crucial for saving lives and ensuring public safety.