如何解决Huggingface Transformer 将 logit 分数转换为概率
我是这个领域的初学者并且被困住了。我正在按照本教程 (https://towardsdatascience.com/multi-label-multi-class-text-classification-with-bert-transformer-and-keras-c6355eccb63a) 使用 Huggingface 变换器构建多标签分类。
以下是我用来训练模型的代码。
# Name of the BERT model to use
model_name = 'bert-base-uncased'
# Max length of tokens
max_length = 100
PATH = 'uncased_L-12_H-768_A-12/'
# Load transformers config and set output_hidden_states to False
config = BertConfig.from_pretrained(PATH)
config.output_hidden_states = False
# Load BERT tokenizer
tokenizer = BertTokenizerFast.from_pretrained(PATH,local_files_only=True,config = config)
# tokenizer = BertTokenizer.from_pretrained(PATH,config = config)
# Load the Transformers BERT model
transformer_model = TFBertModel.from_pretrained(PATH,config = config,from_pt=True)
#######################################
### ------- Build the model ------- ###
# Load the MainLayer
bert = transformer_model.layers[0]
# Build your model input
input_ids = Input(shape=(None,),name='input_ids',dtype='int32')
# attention_mask = Input(shape=(max_length,name='attention_mask',dtype='int32')
# inputs = {'input_ids': input_ids,'attention_mask': attention_mask}
inputs = {'input_ids': input_ids}
# Load the Transformers BERT model as a layer in a Keras model
bert_model = bert(inputs)[1]
dropout = Dropout(config.hidden_dropout_prob,name='pooled_output')
pooled_output = dropout(bert_model,training=False)
# Then build your model output
issue = Dense(units=len(data.U_label.value_counts()),kernel_initializer=Truncatednormal(stddev=config.initializer_range),name='issue')(pooled_output)
outputs = {'issue': issue}
# And combine it all in a model object
model = Model(inputs=inputs,outputs=outputs,name='BERT_MultiLabel_MultiClass')
# Take a look at the model
model.summary()
#######################################
### ------- Train the model ------- ###
# Set an optimizer
optimizer = Adam(
learning_rate=5e-05,epsilon=1e-08,decay=0.01,clipnorm=1.0)
# Set loss and metrics
loss = {'issue': CategoricalCrossentropy(from_logits = True)}
# loss = {'issue': CategoricalCrossentropy()}
metric = {'issue': CategoricalAccuracy('accuracy')}
# Compile the model
model.compile(
optimizer = optimizer,loss = loss,metrics = metric)
from sklearn import preprocessing
le = preprocessing.LabelEncoder()
le.fit(data['U_H_Label'])
# Ready output data for the model
y_issue = to_categorical(le.transform(data['U_H_Label']))
# Tokenize the input (takes some time)
x = tokenizer(
text=data['Input_Data'].to_list(),add_special_tokens=True,max_length=max_length,truncation=True,padding=True,return_tensors='tf',return_token_type_ids = False,return_attention_mask = True,verbose = True)
# Fit the model
history = model.fit(
# x={'input_ids': x['input_ids'],'attention_mask': x['attention_mask']},x={'input_ids': x['input_ids']},y={'issue': y_issue},validation_split=0.2,batch_size=64,epochs=10)
当我使用 model.predict() 函数时,我想我得到了每个类的 logit 分数,并希望将它们转换为 0 到 1 的概率分数。
我在多篇博客中读到过,我必须使用 softmax 函数,但无法说明在哪里以及如何使用。如果有人可以告诉我需要哪行代码,我将不胜感激!
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。