🧀 BigCheese.ai

Social

Octopus v2: On-device language model for super agent

🧀

Octopus v2 is presented as an on-device language model outperforming GPT-4 in accuracy and latency while reducing context length by 95%. The model's 2 billion parameters enhance latency by 35-fold compared to Llama-7B with a RAG-based function calling mechanism, making it suitable for edge devices in real-world applications.

  • Authors are Wei Chen and Zhiyuan Li.
  • The research was submitted on 2 Apr 2024.
  • The method empowers on-device models.
  • Performance surpasses GPT-4.
  • Suitable for a variety of edge devices.