GPT2 감성분석 모형 사용하기 :: 대화형 AI - mindscale
Skip to content

GPT2 감성분석 모형 사용하기

토큰화

from sentencepiece import SentencePieceProcessor

tokenizer = SentencePieceProcessor(model_file='/gdrive/My Drive/kogpt2/kogpt2_news_wiki_ko_cased_818bfa919d.spiece')

text = '즐거운 하루!'

bos = [tokenizer.bos_id()]
body = tokenizer.encode_as_ids(text)
eos = [tokenizer.eos_id()]
tokens = bos + body + eos

tokens

[0, 5671, 1447, 47812, 1]
n = len(tokens) - 1

n

4
tokens = tf.convert_to_tensor([tokens])

GPT2 모형에 입력

from transformers import TFGPT2Model

model = TFGPT2Model.from_pretrained('/gdrive/My Drive/kogpt2/kogpt2_transformers', from_pt=True)

Some weights of the PyTorch model were not used when initializing the TF 2.0 model TFGPT2Model: ['transformer.h.9.attn.masked_bias', 'lm_head.weight', 'transformer.h.1.attn.masked_bias', 'transformer.h.5.attn.masked_bias', 'transformer.h.10.attn.bias', 'transformer.h.3.attn.bias', 'transformer.h.0.attn.masked_bias', 'transformer.h.2.attn.bias', 'transformer.h.3.attn.masked_bias', 'transformer.h.11.attn.bias', 'transformer.h.2.attn.masked_bias', 'transformer.h.7.attn.masked_bias', 'transformer.h.5.attn.bias', 'transformer.h.4.attn.masked_bias', 'transformer.h.8.attn.bias', 'transformer.h.4.attn.bias', 'transformer.h.11.attn.masked_bias', 'transformer.h.6.attn.masked_bias', 'transformer.h.1.attn.bias', 'transformer.h.7.attn.bias', 'transformer.h.6.attn.bias', 'transformer.h.9.attn.bias', 'transformer.h.8.attn.masked_bias', 'transformer.h.10.attn.masked_bias', 'transformer.h.0.attn.bias']
- This IS expected if you are initializing TFGPT2Model from a PyTorch model trained on another task or with another architecture (e.g. initializing a TFBertForSequenceClassification model from a BertForPretraining model).
- This IS NOT expected if you are initializing TFGPT2Model from a PyTorch model that you expect to be exactly identical (e.g. initializing a TFBertForSequenceClassification model from a BertForSequenceClassification model).
All the weights of TFGPT2Model were initialized from the PyTorch model.
If your task is similar to the task the model of the ckeckpoint was trained on, you can already use TFGPT2Model for predictions without further training.

result = model(tokens)

logits = result[0]
logits

<tf.Tensor: shape=(1, 5, 768), dtype=float32, numpy=
array([[[-0.09444575, -0.05083504, -0.13943346, ..., -0.0137838 ,
          0.09048785,  0.06247889],
        [ 0.1413924 , -0.4065953 ,  0.07182949, ..., -0.01347636,
          0.4901314 ,  0.08847989],
        [-0.2025207 , -0.04837355,  0.3766763 , ..., -0.01326111,
          0.13881025, -0.22728492],
        [-0.48055494, -0.0605754 ,  0.01404438, ..., -0.0135248 ,
         -0.12486423, -0.23265909],
        [-0.06227629,  0.00658476, -0.02954386, ..., -0.0143476 ,
          0.06194749,  0.03551652]]], dtype=float32)>
output = tf.gather_nd(logits, [(0, n)])

감성분석 모형에 입력

classifier(output)

<tf.Tensor: shape=(1, 3), dtype=float32, numpy=array([[0.47575575, 0.47491515, 0.04932913]], dtype=float32)>