Commits

Commit history

1. Commits on Aug 4, 2023

   • Include more batchsizes in the test

     Show description for 75f9eff
     

     yangyang22

     committed
     75f9effView commit details

     Copy full SHA for 75f9eff

     Browse repository at this point
2. Commits on Aug 2, 2023

   • Improved output format for better readability

     Show description for f258c21
     

     yangyang22

     committed
     f258c21View commit details

     Copy full SHA for f258c21

     Browse repository at this point
3. Commits on Aug 1, 2023

   • benchmark backward pass using cuda graph

     Show description for d040c73
     

     yangyang22

     committed
     d040c73View commit details

     Copy full SHA for d040c73

     Browse repository at this point
4. Commits on Jul 31, 2023

   • use cuda graph for benchmarking

     Show description for bf83daf
     

     yangyang22

     committed
     bf83dafView commit details

     Copy full SHA for bf83daf

     Browse repository at this point
5. Commits on Jul 20, 2023

   • Force synchronization before measuring time

     Show description for 26c114f
     

     yangyang22

     committed
     26c114fView commit details

     Copy full SHA for 26c114f

     Browse repository at this point

   • Repeat forward and backward passes only in the benchmarking experiments

     Show description for 9e767a0
     

     yangyang22

     committed
     9e767a0View commit details

     Copy full SHA for 9e767a0

     Browse repository at this point

   • Benchmarking fMHA in Transformer Engine

     Show description for 3762dd8
     

     yangyang22

     committed
     3762dd8View commit details

     Copy full SHA for 3762dd8

     Browse repository at this point
6. Commits on Jul 19, 2023

   • FlashAttention 2.0 support (#329)

     Show description for faefacb
     ksivaman

     authored
     faefacbView commit details

     Copy full SHA for faefacb

     Browse repository at this point

   • Fix numerics for activation recompute (#327)

     Show description for ded8b9b
     ksivaman

     authored
     ded8b9bView commit details

     Copy full SHA for ded8b9b

     Browse repository at this point

   • FA does not support head_dim > 64 on Ada (#328)

     Show description for 2a81e93
     ksivaman

     authored
     2a81e93View commit details

     Copy full SHA for 2a81e93

     Browse repository at this point
7. Commits on Jul 18, 2023

   • [JAX] Fully remove attn_type and set self_attn_mask_type default to 'causal' (#324)

     Show description for a3e4e61
     

     zlsh80826

     and

     ksivaman

     authored
     a3e4e61View commit details

     Copy full SHA for a3e4e61

     Browse repository at this point

   • layer number bug fixes (#326)

     Show description for 21162d0
     ksivaman

     authored
     21162d0View commit details

     Copy full SHA for 21162d0

     Browse repository at this point
8. Commits on Jul 17, 2023

   • Change version to 0.12.0dev

     Show description for 706d7da
     ksivaman

     committed
     706d7daView commit details

     Copy full SHA for 706d7da

     Browse repository at this point
9. Commits on Jul 15, 2023

   • Disable TorchDynamo optimizations in PyTorch modules (#312)

     Show description for c6538d6
     

     timmoon10

     and

     ksivaman

     authored
     c6538d6View commit details

     Copy full SHA for c6538d6

     Browse repository at this point

   • Stop creating pip wheels in GitHub build tests (#315)

     Show description for 36873ec
     

     timmoon10

     and

     ksivaman

     authored
     36873ecView commit details

     Copy full SHA for 36873ec

     Browse repository at this point
10. Commits on Jul 14, 2023

    • [PyTorch] Fix FP8 checkpointing for non forward execution cases (#323)

      Show description for f351191
      ksivaman

      authored
      f351191View commit details

      Copy full SHA for f351191

      Browse repository at this point

    • Fix calibration in PyTorch example (#322)

      Show description for 04822f4
      ksivaman

      authored
      04822f4View commit details

      Copy full SHA for 04822f4

      Browse repository at this point

    • Fused attention fixes for cuDNN 8.9.3 (#311)

      Show description for 0707552
      cyanguwa

      authored
      0707552View commit details

      Copy full SHA for 0707552

      Browse repository at this point

    • Deprecate unused APIs (#321)

      Show description for 58d2eba
      ksivaman

      authored
      58d2ebaView commit details

      Copy full SHA for 58d2eba

      Browse repository at this point
11. Commits on Jul 13, 2023

    • Fix dtype for KV inference cache (#319)

      Show description for b172bad
      ksivaman

      authored
      b172badView commit details

      Copy full SHA for b172bad

      Browse repository at this point

    • Fix FP32 LayerNorm ONNX export (#313)

      Show description for 6bd35bf
      

      nzmora-nvidia

      and

      ksivaman

      authored
      6bd35bfView commit details

      Copy full SHA for 6bd35bf

      Browse repository at this point

    • Remove buffer registration for FSDP like cases (#318)

      Show description for 11c5d58
      ksivaman

      authored
      11c5d58View commit details

      Copy full SHA for 11c5d58

      Browse repository at this point

    • Catch cublas FP8 errors (#317)

      Show description for 8c3110d
      ksivaman

      authored
      8c3110dView commit details

      Copy full SHA for 8c3110d

      Browse repository at this point
12. Commits on Jul 7, 2023

    • [JAX] Support arbitrary dimensinos of fp8 meta. (#309)

      Show description for a7bc7cf
      mingxu1067

      authored
      a7bc7cfView commit details

      Copy full SHA for a7bc7cf

      Browse repository at this point
13. Commits on Jul 6, 2023

    • Add operators for Paddle (#285)

      Show description for a7a1a07
      Wong4j

      authored
      a7a1a07View commit details

      Copy full SHA for a7a1a07

      Browse repository at this point
14. Commits on Jul 1, 2023

    • Check for cuDNN frontend API when building (#307)

      Show description for a83605d
      

      timmoon10

      and

      ksivaman

      authored
      a83605dView commit details

      Copy full SHA for a83605d

      Browse repository at this point
15. Commits on Jun 30, 2023

    • Fix TE discovery in python virtual environments (#303)

      Show description for 5976fe1
      tp5uiuc

      authored
      5976fe1View commit details

      Copy full SHA for 5976fe1

      Browse repository at this point
16. Commits on Jun 27, 2023

    • Add FP8 paper (#301)

      Show description for 6f7853d
      ksivaman

      authored
      6f7853dView commit details

      Copy full SHA for 6f7853d

      Browse repository at this point
17. Commits on Jun 26, 2023

    • Get default dtype from pytorch (#300)

      Show description for 79a78ca
      ksivaman

      authored
      79a78caView commit details

      Copy full SHA for 79a78ca

      Browse repository at this point

    • Test ONNX export - missing BF16 GEMM tests + output.json fix (#297)

      Show description for 6bccc76
      

      galagam

      and

      ksivaman

      authored
      6bccc76View commit details

      Copy full SHA for 6bccc76

      Browse repository at this point

    • Fix compute type for GEMM (#296)

      Show description for c56646e
      ksivaman

      authored
      c56646eView commit details

      Copy full SHA for c56646e

      Browse repository at this point
18. Commits on Jun 23, 2023

    • Fix layer_norm ONNX export (#293)

      Show description for 574f1b4
      

      nzmora-nvidia

      and

      ksivaman

      authored
      574f1b4View commit details

      Copy full SHA for 574f1b4

      Browse repository at this point
19. Commits on Jun 22, 2023

    • Add more TE operators for Paddle (#262)

      Show description for 4daea52
      

      Tom-Zheng

      and

      ksivaman

      authored
      4daea52View commit details

      Copy full SHA for 4daea52

      Browse repository at this point

    • Add long sequence support for fused attention (#237)

      Show description for 5c58bea
      

      cyanguwa

      and

      ksivaman

      authored
      5c58beaView commit details

      Copy full SHA for 5c58bea

      Browse repository at this point

    • Update README.rst - Integrations section (#292)

      Show description for 4330e02
      sbhavani

      authored
      4330e02View commit details

      Copy full SHA for 4330e02

      Browse repository at this point