Skip to content

Latest commit

 

History

History

Transformer

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 
 
 
 
 

README.md

Transformer from Scratch

This repository contains a PyTorch implementation of a Transformer model, inspired by the original paper Attention Is All You Need (2017). The model is trained from scratch on an English-to-German dataset using only basic PyTorch components, with custom implementations of multi-head attention, positional encoding, and encoder-decoder architecture.

Features

  • Encoder-decoder Transformer (no external libraries)
  • Custom attention, positional encoding, and training loop
  • Character-level tokenization
  • Outputs valid German sequences

Project Structure

src/
├── model/
├── data/
notebooks/
outputs/