# Decoder\_start\_token\_id per sample or per batch during training

**URL:** <https://discuss.huggingface.co/t/decoder-start-token-id-per-sample-or-per-batch-during-training/73526>\
**Category:** 🤗Transformers\
**Created:** [February 16, 2024, 7:57pm UTC](https://discuss.huggingface.co/t/decoder-start-token-id-per-sample-or-per-batch-during-training/73526 "2024-02-16T19:57:29Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![tehranixyz](https://avatars.discourse-cdn.com/v4/letter/t/a9adbd/32.png) [@tehranixyz](https://discuss.huggingface.co/u/tehranixyz)\
**Post date:** [February 16, 2024, 7:57pm UTC](https://discuss.huggingface.co/t/decoder-start-token-id-per-sample-or-per-batch-during-training/73526/1 "2024-02-16T19:57:29Z")

</div>

Hi All,

I’m working on a Translation model using EncoderDecoderModel.  
Each batch contains samples only from one language.  
How can we define `decoder_start_token_id` per each batch?  
Sometimes, `decoder_start_token_id` is `<ENG>` and sometimes it’s `<FRE>`  
I use `model.module.config.decoder_start_token_id` to change `decoder_start_token_id` per each batch. But this is causing lots of inconsistency when using Multiple GPUs and DeepSpeed.  
Any suggestions?  
It seems `forward` function does not support `decoder_start_token_id`. Only `generate` function supports it. However, I need to specify `decoder_start_token_id` per each batch during training.  
Any suggestions?

Thank you