如何解决了解Trax中的变压器入门示例
我是机器翻译和Trax的新手。我的目标是了解Trax中的变压器入门示例,可以在https://trax-ml.readthedocs.io/en/latest/notebooks/trax_intro.html中找到:
import trax
# Create a Transformer model.
# Pre-trained model config in gs://trax-ml/models/translation/ende_wmt32k.gin
model = trax.models.Transformer(
input_vocab_size=33300,d_model=512,d_ff=2048,n_heads=8,n_encoder_layers=6,n_decoder_layers=6,max_len=2048,mode='predict')
# Initialize using pre-trained weights.
model.init_from_file('gs://trax-ml/models/translation/ende_wmt32k.pkl.gz',weights_only=True)
# Tokenize a sentence.
sentence = 'It is nice to learn new things today!'
tokenized = list(trax.data.tokenize(iter([sentence]),# Operates on streams.
vocab_dir='gs://trax-ml/vocabs/',vocab_file='ende_32k.subword'))[0]
# Decode from the Transformer.
tokenized = tokenized[None,:] # Add batch dimension.
tokenized_translation = trax.supervised.decoding.autoregressive_sample(
model,tokenized,temperature=0.0) # Higher temperature: more diverse results.
# De-tokenize,tokenized_translation = tokenized_translation[0][:-1] # Remove batch and EOS.
translation = trax.data.detokenize(tokenized_translation,vocab_dir='gs://trax-ml/vocabs/',vocab_file='ende_32k.subword')
print(translation)
该示例运行良好。但是,当我尝试使用已初始化模型(例如
)转换另一个示例时sentence = 'I would like to try another example.'
tokenized = list(trax.data.tokenize(iter([sentence]),vocab_file='ende_32k.subword'))[0]
tokenized = tokenized[None,:]
tokenized_translation = trax.supervised.decoding.autoregressive_sample(
model,temperature=0.0)
tokenized_translation = tokenized_translation[0][:-1]
translation = trax.data.detokenize(tokenized_translation,vocab_file='ende_32k.subword')
print(translation)
我在本地计算机和Google Colab上都得到了输出!
。其他示例也是如此。
当我建立并初始化新模型时,一切正常。
这是一个错误吗?如果不是,那么有人会这么向我解释这里发生了什么以及如何避免/修复这种行为?
令牌化和去令牌化似乎运行良好,我对此进行了调试。 trax.supervised.decoding.autoregressive_sample
中似乎出现了错误/意外。
解决方法
我自己发现了这个问题……需要重置模型的state
。因此,以下代码对我有用:
def translate(model,sentence,vocab_dir,vocab_file):
empty_state = model.state # save empty state
tokenized_sentence = next(trax.data.tokenize(iter([sentence]),vocab_dir=vocab_dir,vocab_file=vocab_file))
tokenized_translation = trax.supervised.decoding.autoregressive_sample(
model,tokenized_sentence[None,:],temperature=0.0)[0][:-1]
translation = trax.data.detokenize(tokenized_translation,vocab_file=vocab_file)
model.state = empty_state # reset state
return translation
# Create a Transformer model.
# Pre-trained model config in gs://trax-ml/models/translation/ende_wmt32k.gin
model = trax.models.Transformer(input_vocab_size=33300,d_model=512,d_ff=2048,n_heads=8,n_encoder_layers=6,n_decoder_layers=6,max_len=2048,mode='predict')
# Initialize using pre-trained weights.
model.init_from_file('gs://trax-ml/models/translation/ende_wmt32k.pkl.gz',weights_only=True)
print(translate(model,'It is nice to learn new things today!',vocab_dir='gs://trax-ml/vocabs/',vocab_file='ende_32k.subword'))
print(translate(model,'I would like to try another example.',vocab_file='ende_32k.subword'))
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。