Llama 3 70B Instruct

Llama 3 70B Instruct - Open-Weight Instruction-Tuned Dialogue Model

Llama 3 70B Instruct is Meta's 70-billion parameter open-weight generative text model, instruction-tuned specifically for dialogue and assistant-like chat tasks. It is part of the Llama 3 family released in 8B and 70B sizes.

Training Methodology

The instruction-tuned variant is aligned using supervised fine-tuning (SFT) followed by reinforcement learning with human feedback (RLHF). Fine-tuning data includes publicly available instruction datasets plus over 10 million human-annotated examples. Pretraining used over 15 trillion tokens from publicly available sources, with a data freshness cutoff of December 2023 for the 70B model.

Training was conducted on Meta's Research SuperCluster and production clusters using custom training libraries.

Intended Use

Instruction-tuned models are explicitly scoped to assistant-like chat use cases. Pretrained base variants support broader natural language generation tasks. Out-of-scope use includes languages other than English (fine-tuning for other languages is permitted under the Llama 3 Community License).