> For the complete documentation index, see [llms.txt](https://confeitohs.gitbook.io/gitbook/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://confeitohs.gitbook.io/gitbook/nlp/attention-is-all-you-need-2017.md).

# Attention is All You Need(2017)

Original Paper - Attention Is All You Need (<https://arxiv.org/abs/1706.03762>)

## Abstract

지배적으로 많이 사용되는 시퀀스 변환 모델은 인코더와 디코더를 포함하며 복잡한 Recurrent / Convolutional NN으로 구성된다. 가장 좋은 성능을 내는 모델도 인코더와 디코더를 attention mechanism으로 연결하는 구조이다.

이 논문에선 Recurrent/Convolutional 구조를 완전히 없애고 Attention mechanism에 기반한, 새롭고 간단한 네트워크 구조 - Transformer를 소개한다. 기계번역 상의 성능을 비교했을 때 더 병렬화 쉽고, 적게 train해도 더 좋은 성능을 내었다.

-수치적 성능 내용 생략-

또한 Transformer가 다른 작업에서도 잘 일반화됨을 보였다.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://confeitohs.gitbook.io/gitbook/nlp/attention-is-all-you-need-2017.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.