DL history

Key of DL

• DATA
• MODEL
• LOSS
• OPTIMIZER

2012 - AlexNet

• Deep learning paradigm 시작

2013 - DQN

• 알파고

2014 - Encoder/Decoder, Adam

• seq2seq, machine translation
• 쉬운 구현, 효율적 연산

2015 - GAN, ResNet

• generator, discriminator
• more deep network

2017 - Transformer

• Attention

2018 - BERT

• fine - tuned NLP model

2019 - BIG Language Models

• GPT-X, many parameter

2020 - Self Supervised Learning

• 학습 데이터 외의 데이터셋 활용