fastNLP.models.bert¶

fastNLP提供了BERT应用到五个下游任务的模型代码，可以直接调用。这五个任务分别为

文本分类任务： BertForSequenceClassification

Matching任务： BertForSentenceMatching

多选任务： BertForMultipleChoice

序列标注任务： BertForTokenClassification

抽取式QA任务： BertForQuestionAnswering

每一个模型必须要传入一个名字为 embed 的 fastNLP.embeddings.BertEmbedding ，这个参数包含了 fastNLP.modules.encoder.BertModel ，是下游模型的编码器(encoder)。

除此以外，还需要传入一个数字，这个数字在不同下游任务模型上的意义如下:

下游任务模型                     参数名称      含义
BertForSequenceClassification  num_labels  文本分类类别数目，默认值为2
BertForSentenceMatching        num_labels  Matching任务类别数目，默认值为2
BertForMultipleChoice          num_choices 多选任务选项数目，默认值为2
BertForTokenClassification     num_labels  序列标注标签数目，无默认值
BertForQuestionAnswering       num_labels  抽取式QA列数，默认值为2(即第一列为start_span, 第二列为end_span)

最后还可以传入dropout的大小，默认值为0.1。

class fastNLP.models.bert.BertForSequenceClassification(embed: fastNLP.embeddings.bert_embedding.BertEmbedding, num_labels: int = 2, dropout=0.1)[源代码]¶

基类 fastNLP.models.BaseModel

别名 fastNLP.models.BertForSequenceClassification fastNLP.models.bert.BertForSequenceClassification

BERT model for classification.

__init__(embed: fastNLP.embeddings.bert_embedding.BertEmbedding, num_labels: int = 2, dropout=0.1)[源代码]¶

参数

embed (fastNLP.embeddings.BertEmbedding) -- 下游模型的编码器(encoder).
num_labels (int) -- 文本分类类别数目，默认值为2.
dropout (float) -- dropout的大小，默认值为0.1.

forward(words)[源代码]¶: 输入为 [[w1, w2, w3, ...], [...]], BERTEmbedding会在开头和结尾额外加入[CLS]与[SEP] :param torch.LongTensor words: [batch_size, seq_len] :return: { fastNLP.Const.OUTPUT : logits}: torch.Tensor [batch_size, num_labels]

predict(words)[源代码]¶

参数: words (torch.LongTensor) -- [batch_size, seq_len]
返回: { fastNLP.Const.OUTPUT : logits}: torch.LongTensor [batch_size]

training: bool¶

class fastNLP.models.bert.BertForSentenceMatching(embed: fastNLP.embeddings.bert_embedding.BertEmbedding, num_labels: int = 2, dropout=0.1)[源代码]¶

基类 fastNLP.models.BaseModel

别名 fastNLP.models.BertForSentenceMatching fastNLP.models.bert.BertForSentenceMatching

BERT model for sentence matching.

__init__(embed: fastNLP.embeddings.bert_embedding.BertEmbedding, num_labels: int = 2, dropout=0.1)[源代码]¶

参数

embed (fastNLP.embeddings.BertEmbedding) -- 下游模型的编码器(encoder).
num_labels (int) -- Matching任务类别数目，默认值为2.
dropout (float) -- dropout的大小，默认值为0.1.

forward(words)[源代码]¶

输入words的格式为 [sent1] + [SEP] + [sent2]（BertEmbedding会在开头加入[CLS]和在结尾加入[SEP]），输出为batch_size x num_labels

参数: words (torch.LongTensor) -- [batch_size, seq_len]
返回: { fastNLP.Const.OUTPUT : logits}: torch.Tensor [batch_size, num_labels]

predict(words)[源代码]¶

参数: words (torch.LongTensor) -- [batch_size, seq_len]
返回: { fastNLP.Const.OUTPUT : logits}: torch.LongTensor [batch_size]

training: bool¶

class fastNLP.models.bert.BertForMultipleChoice(embed: fastNLP.embeddings.bert_embedding.BertEmbedding, num_choices=2, dropout=0.1)[源代码]¶

基类 fastNLP.models.BaseModel

别名 fastNLP.models.BertForMultipleChoice fastNLP.models.bert.BertForMultipleChoice

BERT model for multiple choice.

__init__(embed: fastNLP.embeddings.bert_embedding.BertEmbedding, num_choices=2, dropout=0.1)[源代码]¶

参数

embed (fastNLP.embeddings.BertEmbedding) -- 下游模型的编码器(encoder).
num_choices (int) -- 多选任务选项数目，默认值为2.
dropout (float) -- dropout的大小，默认值为0.1.

forward(words)[源代码]¶

参数: words (torch.LongTensor) -- [batch_size, num_choices, seq_len]
返回: { fastNLP.Const.OUTPUT : logits}: torch.LongTensor [batch_size, num_choices]

predict(words)[源代码]¶

参数: words (torch.LongTensor) -- [batch_size, num_choices, seq_len]
返回: { fastNLP.Const.OUTPUT : logits}: torch.LongTensor [batch_size]

training: bool¶

class fastNLP.models.bert.BertForTokenClassification(embed: fastNLP.embeddings.bert_embedding.BertEmbedding, num_labels, dropout=0.1)[源代码]¶

基类 fastNLP.models.BaseModel

别名 fastNLP.models.BertForTokenClassification fastNLP.models.bert.BertForTokenClassification

BERT model for token classification.

__init__(embed: fastNLP.embeddings.bert_embedding.BertEmbedding, num_labels, dropout=0.1)[源代码]¶

参数

embed (fastNLP.embeddings.BertEmbedding) -- 下游模型的编码器(encoder).
num_labels (int) -- 序列标注标签数目，无默认值.
dropout (float) -- dropout的大小，默认值为0.1.

forward(words)[源代码]¶

参数: words (torch.LongTensor) -- [batch_size, seq_len]
返回: { fastNLP.Const.OUTPUT : logits}: torch.Tensor [batch_size, seq_len, num_labels]

predict(words)[源代码]¶

参数: words (torch.LongTensor) -- [batch_size, seq_len]
返回: { fastNLP.Const.OUTPUT : logits}: torch.LongTensor [batch_size, seq_len]

training: bool¶

class fastNLP.models.bert.BertForQuestionAnswering(embed: fastNLP.embeddings.bert_embedding.BertEmbedding)[源代码]¶

基类 fastNLP.models.BaseModel

别名 fastNLP.models.BertForQuestionAnswering fastNLP.models.bert.BertForQuestionAnswering

用于做Q&A的Bert模型，如果是Squad2.0请将BertEmbedding的include_cls_sep设置为True，Squad1.0或CMRC则设置为False

__init__(embed: fastNLP.embeddings.bert_embedding.BertEmbedding)[源代码]¶

参数

embed (fastNLP.embeddings.BertEmbedding) -- 下游模型的编码器(encoder).
num_labels (int) -- 抽取式QA列数，默认值为2(即第一列为start_span, 第二列为end_span).

forward(words)[源代码]¶

输入words为question + [SEP] + [paragraph]，BERTEmbedding在之后会额外加入开头的[CLS]和结尾的[SEP]. note:: 如果BERTEmbedding中include_cls_sep=True，则输出的start和end index相对输入words会增加一位；如果为BERTEmbedding中 include_cls_sep=False, 则输出start和end index的位置与输入words的顺序完全一致

参数: words (torch.LongTensor) -- [batch_size, seq_len]
返回: 一个包含num_labels个logit的dict，每一个logit的形状都是[batch_size, seq_len + 2]

training: bool¶