๋ณธ๋ฌธ ๋ฐ”๋กœ๊ฐ€๊ธฐ

low temperature1

[Notable] Low Temperature Samples Low Temperature Samples๋Š” ์ƒ์„ฑ ๋ชจ๋ธ์—์„œ ์ƒ˜ํ”Œ ํ’ˆ์งˆ์„ ๋†’์ด๊ณ  ๋‹ค์–‘์„ฑ์„ ์ค„์ด๋Š” ๊ธฐ๋ฒ•์ž…๋‹ˆ๋‹ค.์ฃผ๋กœ ํ™•๋ฅ  ๋ถ„ํฌ์˜ "์ƒ˜ํ”Œ๋ง ์˜จ๋„(temperature)"๋ฅผ ์กฐ์ ˆํ•˜์—ฌ ์ƒ์„ฑ ๊ฒฐ๊ณผ์— ์˜ํ–ฅ์„ ์ค๋‹ˆ๋‹ค. 1. "Temperature"์˜ ์˜๋ฏธTemperature๋Š” ํ™•๋ฅ  ๋ถ„ํฌ์˜ "๋‚ ์นด๋กœ์›€(sharpness)" ๋˜๋Š” "๋ถˆํ™•์‹ค์„ฑ(uncertainty)"์„ ์กฐ์ ˆํ•˜๋Š” ํ•˜์ดํผํŒŒ๋ผ๋ฏธํ„ฐ์ž…๋‹ˆ๋‹ค.์ˆ˜ํ•™์ ์œผ๋กœ๋Š” ์†Œํ”„ํŠธ๋งฅ์Šค(softmax) ํ•จ์ˆ˜์—์„œ ์ž์ฃผ ์‚ฌ์šฉ๋ฉ๋‹ˆ๋‹ค:์—ฌ๊ธฐ์„œ:T = temperature (์˜จ๋„)zi = ๋กœ์ง“(logit) ๊ฐ’ (๋ชจ๋ธ์ด ์˜ˆ์ธกํ•œ ์ ์ˆ˜)P(xi) = ์ตœ์ข… ํ™•๋ฅ  2. Temperature์˜ ์˜ํ–ฅ๋†’์€ ์˜จ๋„ (Tโ‰ซ1)ํ™•๋ฅ  ๋ถ„ํฌ๊ฐ€ **ํ‰ํ‰(flat)**ํ•ด์ง€๊ณ , ๋” ๋‹ค์–‘ํ•œ ์ƒ˜ํ”Œ์ด ์ƒ์„ฑ๋จ๋ชจ๋ธ์ด ๋ถˆํ™•์‹คํ•œ ์„ ํƒ์„ ๋” ๋งŽ์ด .. 2025. 2. 5.

ํ‹ฐ์Šคํ† ๋ฆฌํˆด๋ฐ”