# Generating decoder input ids during inference for opus-mt

**URL:** <https://discuss.huggingface.co/t/generating-decoder-input-ids-during-inference-for-opus-mt/9471>\
**Category:** 🤗Transformers\
**Created:** [August 25, 2021, 6:28pm UTC](https://discuss.huggingface.co/t/generating-decoder-input-ids-during-inference-for-opus-mt/9471 "2021-08-25T18:28:43Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![HarshitaDiddee](https://avatars.discourse-cdn.com/v4/letter/h/3d9bf3/32.png) [@HarshitaDiddee](https://discuss.huggingface.co/u/HarshitaDiddee)\
**Post date:** [August 25, 2021, 6:28pm UTC](https://discuss.huggingface.co/t/generating-decoder-input-ids-during-inference-for-opus-mt/9471/1 "2021-08-25T18:28:43Z")

</div>

Hello,

Goal: I am trying to run an inference on the tensorflow lite variant of the opus-mt (en-hi) model.

Question: I just wanted to confirm why is the model supplied decoder\_inputs\_ids during inference and why are they, if I have correctly understood their assignment, being assigned with shifted input id’s. Given that during training - I will be supplying the target\_sequences through this parameter, will this not lead to a kind of inferential inconsistency in the model’s operation ?

Context: During inference, the interpretor requires 3 inputs - the input\_ids, decoder\_input\_ids and the attention\_masks that I have to provide at the edge side. From my understanding, HF’s generate() function already handles the generation of two of the optional inputs i.e decoder\_input\_ids and the attention masks. In my case, I need to supply them mandatorily. I looked through [this](https://github.com/huggingface/transformers/blob/master/src/transformers/models/marian/modeling_flax_marian.py) to get an idea about their generation and inferred them to be -

1. decoder\_input\_ids set as inputs\_ids shifted by the pad\_token\_id ( as specified in HF’s Marian MT Documentation )
2. attention\_masks as a default set attention mask ( naively a numpy.ones(\<batch\_len, max\_seq\_length\>))

Note: I have been able to successfully generate an output using these assignments for the tflite interpretor - So this isn’t particularly a syntactic issue.

Thanks in advance for any help!