• Luc Georges's avatar
    refactor: benchmarks (#33896) · 144852fb
    Luc Georges authored
    
    
    * refactor: benchmarks
    
    Based on a discussion with @LysandreJik & @ArthurZucker, the goal of
    this PR is to improve transformers' benchmark system.
    
    This is a WIP, for the moment the infrastructure required to make things
    work is not ready. Will update the PR description when it is the case.
    
    * feat: add db init in benchmarks CI
    
    * fix: pg_config is missing in runner
    
    * fix: add psql to the runner
    
    * fix: connect info from env vars + PR comments
    
    * refactor: set database as env var
    
    * fix: invalid working directory
    
    * fix: `commit_msg` -> `commit_message`
    
    * fix: git marking checked out repo as unsafe
    
    * feat: add logging
    
    * fix: invalid device
    
    * feat: update grafana dashboard for prod grafana
    
    * feat: add `commit_id` to header table
    
    * feat: commit latest version of dashboard
    
    * feat: move measurements into json field
    
    * feat: remove drop table migration queries
    
    * fix: `torch.arrange` -> `torch.arange`
    
    * fix: add missing `s` to `cache_position` positional argument
    
    * fix: change model
    
    * revert: `cache_positions` -> `cache_position`
    
    * fix: set device for `StaticCache`
    
    * fix: set `StaticCache` dtype
    
    * feat: limit max cache len
    
    * fix script
    
    * raise error on failure!
    
    * not try catch
    
    * try to skip generate compilation
    
    * update
    
    * update docker image!
    
    * update
    
    * update again!@
    
    * update
    
    * updates
    
    * ???
    
    * ??
    
    * use `torch.cuda.synchronize()`
    
    * fix json
    
    * nits
    
    * fix
    
    * fixed!
    
    * f**k
    
    * feat: add TTNT panels
    
    * feat: add try except
    
    ---------
    
    Co-authored-by: default avatarArthur Zucker <arthur.zucker@gmail.com>
    144852fb