attention

    Attention - Query, Key, Value์˜ ์˜๋ฏธ?
    Artificial intelligence/Etc

    Attention - Query, Key, Value์˜ ์˜๋ฏธ?

    Attention์ด ๋‚˜์˜จ ์ด์œ ? Attention์€ LSTM ํ˜น์€ RNN์œผ๋กœ ๋งŒ๋“ค์–ด์ง„ Encoder Decoder์—์„œ ๋‚˜์˜จ ๊ฐœ๋…์ž…๋‹ˆ๋‹ค. Encoder์— ์ž…๋ ฅ์— ๋Œ€ํ•œ ์ •๋ณด๋ฅผ ์€๋‹‰๋ฒกํ„ฐ๋กœ ํ•จ์ถ•์‹œ์ผœ ์ „๋‹ฌ์„ ํ•˜๋Š”๋ฐ, ์ด ์€๋‹‰๋ฒกํ„ฐ๊ฐ€ ๋ชจ๋“  ์ •๋ณด๋ฅผ ๋‹ด๊ธฐ ์–ด๋ ต๋‹ค๋Š” ์ด์œ ์—์„œ ํƒ„์ƒํ•˜๊ฒŒ ๋˜์—ˆ์Šต๋‹ˆ๋‹ค. Decoder์—์„œ ํ•ญ์ƒ Encoder์˜ ๋งˆ์ง€๋ง‰ ์€๋‹‰๋ฒกํ„ฐ๋ฅผ ํ™œ์šฉํ•˜๋Š”๋ฐ, ์ด ์ •๋„์—๋‹ค +๋กœ Encoder์—์„œ ํ˜„์žฌ ์œ„์น˜์™€ ์—ฐ๊ด€๋œ ๋‹จ์–ด์— ๋Œ€ํ•œ ์€๋‹‰๋ฒกํ„ฐ๋ฅผ ๋‹ค์‹œํ•œ๋ฒˆ ๋ด์„œ(์ง‘์ค‘ํ•ด์„œ - Attention) ์€๋‹‰๋ฒกํ„ฐ๋กœ ํ‘œํ˜„ํ•˜์ง€ ๋ชปํ•œ ๋ถ€๋ถ„๊นŒ์ง€ ๋ณด๋ ค๋Š” ๋…ธ๋ ฅ์„ ํ•˜๊ฒŒ๋ฉ๋‹ˆ๋‹ค. Attention์˜ ์ดํ•ด ์–ดํ…์…˜์ด ๊ณ„์‚ฐ๋˜๋Š” ๊ณผ์ •์„ ๊ฐ„๋‹จํžˆ ์„ค๋ช…๋“œ๋ฆฌ๊ณ  ์šฐ๋ฆฌ๊ฐ€ ๊ถ๊ธˆํ•œ Query, Key, Value๋ฅผ ์•Œ์•„๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค. Encoder์—์„œ ๋“ค์–ด์˜จ ๊ฐ’์€ I am a s..

ํ‹ฐ์Šคํ† ๋ฆฌํˆด๋ฐ”