- 14 Apr, 2025 18 commits
-
-
Parteek authored
Co-authored-by:
Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com>
-
Yann Chéné authored
* Add ImageProcessorFast to BiT processor * propose a fast processor and add tests * all tests pass except one * run make * remove useless print * use same test as clip * apply make * Update src/transformers/models/bit/image_processing_bit_fast.py Co-authored-by:
Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com> * Update setup.py Co-authored-by:
Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com> * Update src/transformers/models/bit/image_processing_bit_fast.py Co-authored-by:
Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com> * apply review comment --------- Co-authored-by:
Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com>
-
Parteek authored
* Add Fast LeViT Processor * Update levit.md * Update src/transformers/models/levit/image_processing_levit_fast.py Co-authored-by:
Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com> * ruff check --------- Co-authored-by:
Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com>
-
Rupesh K Srivastava authored
* fix BlockMask handling when using flex_attention for llama/mistral/gemma2 * fix attention_mask types * revert type hints and fixup * remove unnecessary assertion
-
Keumgang Cha authored
* Update modeling_deta.py * variable initialization
-
Vinh H. Pham authored
* add donut fast image processor support * run make style * Update src/transformers/models/donut/image_processing_donut_fast.py Co-authored-by:
Parteek <parteekkamboj112@gmail.com> * update test, remove none default values * add do_align_axis = True test, fix bug in slow image processor * run make style * remove np usage * make style * Apply suggestions from code review * Update src/transformers/models/donut/image_processing_donut_fast.py Co-authored-by:
Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com> * add size revert in preprocess * make style * fix copies * add test for preprocess with kwargs * make style * handle None input_data_format in align_long_axis --------- Co-authored-by:
Parteek <parteekkamboj112@gmail.com> Co-authored-by:
Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com>
-
Cyril Vallez authored
* Update test_modeling_common.py * Fix Llama and its modular children * Update test_modeling_common.py * qwen3 * first try at prioritizing models * Update test_modeling_common.py * Update test_modeling_common.py * Update test_modeling_common.py * test * fix * fix * more models * more * more * more * smarter init for composite models! * fix post rebase * smol * fix missing args * more * typo * Super elegant and efficient init for submodels * Update modeling_utils.py * style * last fixes * cleanup * finalize cleanup * CIs * improve docstring * Update modeling_utils.py * llama4 * style * CIs * style * add dpt * granite speech * qwen 2.5 omni * better fix * Parse the config file instead * CIs
-
Vinh H. Pham authored
* support fast image processor layoutlmv3 * make style * add warning and update test * make style * Update src/transformers/models/layoutlmv3/image_processing_layoutlmv3_fast.py * Update image_processing_auto.py --------- Co-authored-by:
Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com>
-
Cypher Pepe authored
* Update broken link * Update broken link
-
Vinh H. Pham authored
* add support layoutlmv2 * make style * Apply suggestions from code review Co-authored-by:
Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com> * add warning and clean up * make style * Update src/transformers/models/layoutlmv2/image_processing_layoutlmv2_fast.py Co-authored-by:
Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com> --------- Co-authored-by:
Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com>
-
Vinh H. Pham authored
* support flava fast image processor * run style and quality * update test * update according to reviews * make style * update comment on BICUBIC * make style --------- Co-authored-by:
Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com>
-
Raushan Turganbay authored
happy doc ci
-
Vinh H. Pham authored
* add test and fast image processor * make style * Update src/transformers/models/perceiver/image_processing_perceiver_fast.py Co-authored-by:
Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com> * make style --------- Co-authored-by:
Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com>
-
BakerBunker authored
* Add qwen2.5-omni * Remove einops dependency * Add torchdiffeq dependency * Sort init * Add torchdiffeq to extras['diffeq'] * Fix repo consistency * use cached_file * del odeint * renew pytest * format * Remove torchdiffeq * format * fixed batch infer bug * Change positional_embedding to parameter * Change default speaker * Config revision * Use modular & code clean * code clean * decouple padding with model & code cleaning * sort init * fix * fix * Second code review * fix * fix * rename vars to full name + some comments * update pytest * Code clean & fix * fix * style * more clean up * fixup * smaller vision model in tests * fix processor test * deflake a bit the tests (still flaky though) * de-flake tests finally + add generation mixin * final nits i hope * make sure processor tests are complete * replace with Qwen2_5OmniForConditionalGeneration * fix tests after updating ckpt * fix typos when cleaning, also we can't change ckpt * fixup * images and videos kwargs for processor * thinker and talker loadable from hub ckpt * address comments and update tests after rebase * fixup * skip for now * fixup * fixup * remove torch dependency in processors --------- Co-authored-by:
lvyuanjun.lyj <lvyuanjun.lyj@alibaba-inc.con> Co-authored-by:
feizi.wx <feizi.wx@alibaba-inc.com> Co-authored-by:
raushan <raushan@huggingface.co>
-
Yih-Dar authored
* fix * slow --------- Co-authored-by:
ydshieh <ydshieh@users.noreply.github.com>
-
cyyever authored
Signed-off-by:
cyy <cyyever@outlook.com> Co-authored-by:
Yih-Dar <2521628+ydshieh@users.noreply.github.com>
-
Yao Matrix authored
Signed-off-by:
YAO Matrix <matrix.yao@intel.com> Co-authored-by:
Yih-Dar <2521628+ydshieh@users.noreply.github.com>
-
Mehant Kammakomati authored
fix: fix no_split_modules to be picked up for fsdpv1 and v2 sharding Signed-off-by:
Mehant Kammakomati <mehant.kammakomati2@ibm.com>
-
- 11 Apr, 2025 19 commits
-
-
Eric Wiener authored
* Fix issues * Fix comment --------- Co-authored-by:
Pavel Iakubovskii <qubvel@gmail.com>
-
Joao Gante authored
-
Matt authored
* Delete hubconf.py * Trigger tests
-
Alex Brooks authored
* First pass at speech granite Add encoder / projector, rename things * Combine into one model file with causal lm outputs for forward * Add loss calc * Fix config loading Signed-off-by:
Alex-Brooks <Alex.brooks@ibm.com> * Split new / old loading logic * Use transformers integration for loading peft adapters * Add generation wrapper for selective lora enablement * Add note for qformer encoder automodel * Guard torch/audio imports in feature extractor * Handle granite speech autoclasses * Handle optional deps in package structure for granite speech * Add granite pretrained model def for init * Add dummy objects for torch/torchaudio * Add tests for granite speech processor * Minor formatting fixes and refactoring * Add options for falling back to config in forward * Tentative model docstrings for granite speech * Fix config type * Remove legacy load * Allow non-lora variants for granite speech * Override weight tying for llm * Use text config instead of llm config * Add output embeddings getter to fix weight tying * Fix relative imports * computing the number of audio features, based on the raw audio sequence. * collating audio inputs, and keeping the original lengths. * asserted we have text. otherwise we can't specify the audio special token. * assering the number of audio-symbols/audios match correctly. running get validated_audios only when audio is present * indentation bugfix + supporting different feature lengths when expanding audio. * redundant, done in _get_validated_text * adapting the tests: - we must have text (not either audio or text) - _get_num_audio_features takes a list of raw lengths, provided it insetad. * Minor cleanup, remove unused import * Add more tests for batch feature processing * Allow setting offset in rel position embeddings * Add config option for warning if peft is not installed w/ lora * Port blip2 qformer code into granite speech * Add sad test for numpy arr processing * Allow numpy arrays / tuples in granite speech processor * Fix config type for projector * - pad instead of creating a zeros tensor, to keep the original dtype/device (support bfloat16) - cast input_features to the model dtype (support bfloat16) * merge Blip2QFormerConfig to GraniteSpeechProjectorConfig * prevent a crash when re-saving/loading the model (line 109) * consider additional edge cases during preprocessing. * consider additional edge cases during preprocessing. * add features mask for batched inference (bugfix) * Minor refactor, remove multiaudio processor tests * Add set input/output embeddings for granite speech * Fix feature dim check in processor test * Pop input features in embed test for granite speech * Small fixes for test edge cases Add granite speech to seq2seq causal lm mapping names * Add small tests for granite speech model * Fix data parallelism test * Standardize model class names * Fix check for copies * Fix misaligned init check * Skip granite speech in checkpoint check * Use default for tie_word_embeddings in granite speech * Fix non documentation granite speech repo issues * Fix comments and docstring checks * Add placeholder docs for granite speech * Fix test naming collision * Code formatting * Rerun torch dummy obj regen * Fix save pretrained for granite speech * Import sorting * Fix tests typo * Remove offset hack * Pass args through encoder config * Remove unused prune heads from blip2 * removing einsum. replaced with explicit multiplication (relative positional encodings) and sdpa attention. * remove Sequential from ConformerFeedForward and ConformerConvModule. + fix for sdpa attention * remove GraniteSpeechConformerScale * rename to hidden_states * rename conformer layers to self.layers, remove the first linear from the list to keep the list homogenous. * move pre-norm to the attention/feedforward blocks (avoid complex module wrapping) * adding pre_norm into forward * feature extractor refactoring to resemble how it's done in phi4multimodal. * rename feature_extractor to audio_processor * bugfix: input_feature_mask fix to get the exact number tokens. * Fix pytest decorator in processor test * Add (disabled) integration tests for granite speech * Fix handling of optional feature masking * Loosen validation in processing for vLLM compatability * Formatting fixes * Update init structure to mirror llama * Make granite speech projector generic * Update test config to reflect generic projector * Formatting fixes * Fix typos, add license * Fix undefined var in input processing * Cleanup and expose ctc encoder * Add missing config docstrings * Better var names, type hints, etc * Set attn context size in init * Add max pos emb to encoder config * Cleanup feature extractor * Add granite speech architecture details * Remove granite speech qformer ref * Add paper link, explicit calc for qkv * Calculate padding directly in depthwise conv1d init * Raise value error instead of asserting * Reorder class defs (classes used at top) * Precompute relpos distances * Run formatting * Pass attention distances through forward * Apply suggestions from code review Co-authored-by:
eustlb <94853470+eustlb@users.noreply.github.com> * Add todo for using common batch feature extraction * Rename audios/features * Ensure chat template may be provided to processor * Move granite speech docs to audio models * Add todos for input proc refactoring * Fix import order * Guard torch import * Use relative imports * Require torch backend for processor in granite speech * Add backend guards in feature extractor --------- Signed-off-by:
Alex-Brooks <Alex.brooks@ibm.com> Co-authored-by:
Avihu Dekel <avihu.dekel@ibm.com> Co-authored-by:
eustlb <94853470+eustlb@users.noreply.github.com>
-
Mehant Kammakomati authored
nit: typing to text config Signed-off-by:
Mehant Kammakomati <mehant.kammakomati2@ibm.com>
-
cyyever authored
* Add xpu case to is_torch_bf16_gpu_available Signed-off-by:
cyy <cyyever@outlook.com> * Refine error messages Signed-off-by:
cyy <cyyever@outlook.com> --------- Signed-off-by:
cyy <cyyever@outlook.com> Co-authored-by:
Marc Sun <57196510+SunMarc@users.noreply.github.com>
-
cyyever authored
-
Matt authored
🚨 🚨 Allow saving and loading multiple "raw" chat template files (#36588) * Add saving in the new format (but no loading yet!) * Add saving in the new format (but no loading yet!) * A new approach to template files! * make fixup * make fixup, set correct dir * Some progress but need to rework for cached_file * Rework loading handling again * Small fixes * Looks like it's working now! * make fixup * Working! * make fixup * make fixup * Add TODO so I don't miss it * Cleaner control flow with one less indent * Copy the new logic to processing_utils as well * Proper support for dicts of templates * make fixup * define the file/dir names in a single place * Update the processor chat template reload test as well * Add processor loading of multiple templates * Flatten correctly to match tokenizers * Better support when files are empty sometimes * Stop creating those empty templates * Revert changes now we don't have empty templates * Revert changes now we don't have empty templates * Don't support separate template files on the legacy path * Rework/simplify loading code * Make sure it's always a chat_template key in chat_template.json * Update processor handling of multiple templates * Add a full save-loading test to the tokenizer tests as well * Correct un-flattening * New test was incorrect * Correct error/offline handling * Better exception handling * More error handling cleanup * Add skips for test failing on main * Reorder to fix errors * make fixup * clarify legacy processor file docs and location * Update src/transformers/processing_utils.py Co-authored-by:Lucain <lucainp@gmail.com> * Update src/transformers/processing_utils.py Co-authored-by:
Lucain <lucainp@gmail.com> * Update src/transformers/processing_utils.py Co-authored-by:
Lucain <lucainp@gmail.com> * Update src/transformers/processing_utils.py Co-authored-by:
Lucain <lucainp@gmail.com> * Rename to _jinja and _legacy * Stop saving multiple templates in the legacy format * Cleanup the processing code * Cleanup the processing code more * make fixup * make fixup * correct reformatting * Use correct dir name * Fix import location * Use save_jinja_files instead of save_raw_chat_template_files * Correct the test for saving multiple processor templates * Fix type hint * Update src/transformers/utils/hub.py Co-authored-by:
Julien Chaumond <julien@huggingface.co> * Patch llava_onevision test * Update src/transformers/processing_utils.py Co-authored-by:
Julien Chaumond <julien@huggingface.co> * Update src/transformers/tokenization_utils_base.py Co-authored-by:
Julien Chaumond <julien@huggingface.co> * Refactor chat template saving out into a separate function * Update tests for the new default * Don't do chat template saving logic when chat template isn't there * Ensure save_jinja_files is propagated to tokenizer correctly * Trigger tests * Update more tests to new default * Trigger tests --------- Co-authored-by:
Lucain <lucainp@gmail.com> Co-authored-by:
Julien Chaumond <julien@huggingface.co>
-
Mohamed Mekkouri authored
fix
-
Wing Lian authored
prevent creating a view/leaf param for low rank optimizers:
-
Bowen Bao authored
-
Raushan Turganbay authored
* clkea up mulitmodal processor tests * fixup * fix tests * fix one last test * forgot
-
Mohamed Mekkouri authored
* remove mlp for now * disable on docker
-
Lysandre Debut authored
Test fetcher
-
Arthur authored
* the fix that did not get in * add kernels * full graph does not work * simpler is better * Update src/transformers/integrations/hub_kernels.py Co-authored-by:
Daniël de Kok <me@danieldk.eu> * Update src/transformers/integrations/fbgemm_fp8.py Co-authored-by:
Daniël de Kok <me@danieldk.eu> * Update src/transformers/integrations/hub_kernels.py Co-authored-by:
Daniël de Kok <me@danieldk.eu> * fixup --------- Co-authored-by:
Daniël de Kok <me@danieldk.eu>
-
Arthur authored
* update `kernels` * oups * new pinned version
-
Lysandre Debut authored
* Reverse dependency map shouldn't be created when test_all is set * [test_all] Remove dummies * Modular fixes * Update utils/check_repo.py Co-authored-by:
Pablo Montalvo <39954772+molbap@users.noreply.github.com> * [test_all] Better docs * [test_all] Update src/transformers/commands/chat.py Co-authored-by:
Joao Gante <joaofranciscocardosogante@gmail.com> * [test_all] Remove deprecated AdaptiveEmbeddings from the tests * [test_all] Doc builder * [test_all] is_dummy * [test_all] Import utils * [test_all] Doc building should not require all deps --------- Co-authored-by:
Pablo Montalvo <39954772+molbap@users.noreply.github.com> Co-authored-by:
Joao Gante <joaofranciscocardosogante@gmail.com>
-
Donggeun Yu authored
Corrects the file path used to locate the CUDA kernels for the Deformable Attention module. This ensures that the kernels are loaded correctly, resolving potential errors during module initialization and usage.
-
Yao Matrix authored
* enhance require_deterministic_for_xpu Signed-off-by:
YAO Matrix <matrix.yao@intel.com> * fix style Signed-off-by:
YAO Matrix <matrix.yao@intel.com> * fix style Signed-off-by:
YAO Matrix <matrix.yao@intel.com> --------- Signed-off-by:
YAO Matrix <matrix.yao@intel.com>
-
- 10 Apr, 2025 3 commits
-
-
cyyever authored
* Remove unneeded library version checks Signed-off-by:
cyy <cyyever@outlook.com> * Remove PyTorch condition Signed-off-by:
cyy <cyyever@outlook.com> * Remove PyTorch condition Signed-off-by:
cyy <cyyever@outlook.com> * Fix ROCm get_device_capability Signed-off-by:
cyy <cyyever@outlook.com> * Revert "Fix ROCm get_device_capability" This reverts commit 0e756434bd7e74ffd73de5500476072b096570a6. * Remove unnecessary check Signed-off-by:
cyy <cyyever@outlook.com> * Revert changes Signed-off-by:
cyy <cyyever@outlook.com> --------- Signed-off-by:
cyy <cyyever@outlook.com>
-
duanjunwen authored
-
Mohamed Mekkouri authored
add myself
-