🧀 BigCheese.ai




EleutherAI introduces Pile-T5, a new variant of the T5 model trained on the Pile dataset with the LLAMA tokenizer for improved code-related tasks. Pile-T5 outperforms previous T5 models in various benchmarks, offering vast improvements on code tasks. Researchers can access models and scripts, including intermediate checkpoints for in-depth study.

  • Pile-T5 trained to 2 trillion tokens.
  • Uses LLAMA tokenizer.
  • Outperforms T5-v1.1 models.
  • Available on Huggingface.
  • Evaluated on SuperGLUE & more.