Home

       [HN Gopher] Triton Fork for Windows Support
       ___________________________________________________________________
        
       Triton Fork for Windows Support
        
       Author : lnyan
       Score  : 17 points
       Date   : 2024-10-15 15:40 UTC (1 days ago)
        
  HTML web link (github.com)
  TEXT w3m dump (github.com)
        
       | yjftsjthsd-h wrote:
       | "Triton" here is apparently a programming language, which
       | upstream describes as
       | 
       | > This is the development repository of Triton, a language and
       | compiler for writing highly efficient custom Deep-Learning
       | primitives. The aim of Triton is to provide an open-source
       | environment to write fast code at higher productivity than CUDA,
       | but also with higher flexibility than other existing DSLs.
       | 
       | So if you clicked in expecting the illumos-based virtualization
       | platform, this isn't that. Though
       | 
       | > This is the basis for torchao, which crucially changes some
       | large models from "can't run" to "can run" on consumer GPUs.
       | That's easier than supporting them in other quantization
       | frameworks, or letting the consumers use Linux or WSL
       | 
       | Does sound neat on its own merits.
        
       | Scene_Cast2 wrote:
       | This is pretty great. PyTorch uses triton as the backend for
       | torch.compile (the big feature of PyTorch 2.0, and the necessary
       | part for making Flex Attention in the about to be released 2.5
       | usably fast).
       | 
       | Triton's team doesn't support Windows, and, worse yet, does not
       | accept community PRs to enable any sort of support.
       | 
       | Here's the github issue: https://github.com/triton-
       | lang/triton/issues/1640
       | 
       | And here's the performance comparison of Flex Attention with and
       | without torch.compile (tldr it's 3x slower than a standard MHA
       | when not compiled): https://github.com/rasbt/LLMs-from-
       | scratch/blob/76e9a9ec02a1...
       | 
       | EDIT: after taking a look at the repo, the only thing changed in
       | the "46 commits ahead of [official triton]" is the README.
       | Somewhat sketchy.
        
         | zorgmonkey wrote:
         | It is mentioned at the top of the readme, but the actual code
         | changes appear to be on the branch v3.1.x-windows (also a few
         | other branches with -windows in the name). Also the triton seem
         | willing to collaborate as long as the patches sent are
         | reasonably minimal and high quality https://github.com/triton-
         | lang/triton/pull/4045#issuecomment...
        
       ___________________________________________________________________
       (page generated 2024-10-17 06:01 UTC)