혼란도 계산

모형을 불러온다

import tensorflow as tf
from transformers import TFAutoModelForCausalLM, AutoTokenizer

model = TFAutoModelForCausalLM.from_pretrained('gpt2')
tokenizer = AutoTokenizer.from_pretrained('gpt2')

All model checkpoint weights were used when initializing TFGPT2LMHeadModel.

All the weights of TFGPT2LMHeadModel were initialized from the model checkpoint at gpt2.
If your task is similar to the task the model of the checkpoint was trained on, you can already use TFGPT2LMHeadModel for predictions without further training.

HBox(children=(FloatProgress(value=0.0, description='Downloading', max=1042301.0, style=ProgressStyle(descript…

HBox(children=(FloatProgress(value=0.0, description='Downloading', max=456318.0, style=ProgressStyle(descripti…

문장을 만든다

text = 'I have a car.'

토큰화

input_ids = tokenizer.encode(text, return_tensors='tf')

input_ids

<tf.Tensor: shape=(1, 5), dtype=int32, numpy=array([[  40,  423,  257, 1097,   13]], dtype=int32)>

문장을 모형에 넣는다

output = model(input_ids)

로짓을 구한다

logits = output[0]

혼란도를 계산한다

tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(input_ids, logits))

<tf.Tensor: shape=(), dtype=float32, numpy=8.426451>

문장을 생성한다

input_ids = tokenizer.encode('I have', return_tensors='tf')

result = model.generate(input_ids, max_length=10)

Setting `pad_token_id` to 50256 (first `eos_token_id`) to generate sequence

tokenizer.decode(result[0])

'I have a lot of respect for the people who'

생성된 문장의 혼란도를 계산한다

output = model(result)
logits = output[0]

tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(result[0], logits[0]))

<tf.Tensor: shape=(), dtype=float32, numpy=9.285423>