• Susnato Dhar's avatar
    Add Phi-1 and Phi-1_5 (#26170) · e1c3ac25
    Susnato Dhar authored
    * only dir not even init
    
    * init
    
    * tokenizer removed and reference of codegen added
    
    * modeling file updated a lot remaining app_rotary_emb
    
    * conversion script done
    
    * conversion script fixed, a lot of factoring done and most tests pass
    
    * added token_clf and extractive_QA_head
    
    * integration tests pass
    
    * flash attn tests pass!
    
    * config done
    
    * more docs in modeling file
    
    * some style fix
    
    * style and others
    
    * doc test error fix
    
    * more doc fix
    
    * some attention fixes
    
    * most fixes
    
    * style and other fixes
    
    * docs fix and config
    
    * doc fix
    
    * some comments
    
    * conversion script updated
    
    * conversion script updated
    
    * Revert "conversion script updated"
    
    This reverts commit e92378c54084ec0747041b113083d1746ecb6c7f.
    
    * final comments
    
    * add Phi to language_modeling.md
    
    * edit phi.md file
    
    * rebase and fix
    
    * removed phi-1.5 example
    
    * changed model_type from 'phi'->'mixformer-sequential'
    
    * small change
    
    * small change
    
    * revert \small change
    
    * changed mixformer-sequential->phi
    
    * small change
    
    * added phi-1.5 example instead of phi-1
    
    * doc test might pass now
    
    * rebase and small change
    
    * added the dropout layer
    
    * more fixes
    
    * modified .md file
    
    * very very small doc change
    e1c3ac25