Ehab Saleh | techkahwa.net Model released: May 13, 2024 | Published: May 30, 2024 The numbers first Metric Result What it measures Source Falcon 2 11B on Arabic ARC-C, 25-shot 25.32 A four-option test whose random baseline is 25 Technical report, arXiv:2407.14885, Table 11 On Arabic MMLU, 25-shot 28.04 General knowledge Same table On Arabic … Read more
Ehab Saleh | techkahwa.net Model released: May 13, 2024 | Published: May 27, 2024 The numbers first Metric Result What it measures Source Arabic, before the new tokenizer 53 tokens The same sample sentence GPT-4o launch page, May 13, 2024 Arabic, after the new tokenizer 26 tokens A reduction by a factor of two Same … Read more
Ehab Saleh | techkahwa.net Model released: April 4, 2024 | Published: April 18, 2024 The numbers first Metric Result What it measures Source Arabic among the optimised languages Yes, by name An explicit commitment from a Western lab Model card, Cohere Number of optimised languages 10 English, French, Spanish, Italian, German, Brazilian Portuguese, Japanese, Korean, … Read more
Ehab Saleh | techkahwa.net Model released: March 4, 2024 | Published: March 18, 2024 The numbers first Metric Result What it measures Source Release date March 4, 2024 Opus and Sonnet available the same day Anthropic announcement Multilingual results in the model card Present: MGSM multilingual math, and multilingual MMLU Section 5.6.1 Claude 3 model … Read more
Ehab Saleh | techkahwa.net Paper released: February 20, 2024 | Published: March 5, 2024 The numbers first Metric Result What it measures Source Number of questions 14,575 multiple choice The size of the exam ArabicMMLU paper, Findings of ACL 2024 Number of tasks 40 Breadth of domains Same paper Source countries 8 Arab countries Geographic … Read more
Ehab Saleh | techkahwa.net Model released: December 11, 2023 | Published: December 25, 2023 The numbers first Metric Result What it measures Source Declared languages Five: English, French, Italian, German, Spanish What the lab says about its own model Mistral announcement, December 11, 2023 Arabic mentioned in the announcement Not present Whether it was mentioned … Read more
Ehab Saleh | techkahwa.net Model released: November 6, 2023 | Published: November 20, 2023 The numbers first Metric Result What it measures Source Whisper large-v3, Modern Standard Arabic on the SADA set 27.95% error The model’s best case in Arabic Open Universal Arabic ASR Leaderboard, Interspeech 2025 Whisper large-v3, Najdi 48.58% The first stage of … Read more
Ehab Saleh | techkahwa.net Paper released: September 21, 2023 | Published: October 5, 2023 The numbers first Metric Result What it measures Source AceGPT-13B-chat on Arabic Vicuna-80 100.88% of GPT-3.5’s performance Arabic instruction following, judged by GPT-4 AceGPT paper, NAACL 2024 AceGPT-13B-chat on Arabic AlpacaEval 97.95% of GPT-3.5 Same measure, different set Same paper Jais-13B-chat … Read more
Ehab Saleh | techkahwa.net Model released: September 6, 2023 | Published: September 20, 2023 The numbers first Metric Result What it measures Source Model size 180 billion parameters The largest open model of its moment Technology Innovation Institute press release, September 6, 2023 Training data 3.5 trillion tokens Also the largest Same release Languages stated … Read more
Ehab Saleh | techkahwa.net Model released: August 30, 2023 | Published: September 13, 2023 The numbers first Metric Result What it measures Source Total training tokens 395 billion The model’s overall scale Jais paper, arXiv:2308.16149, August 2023 Original Arabic 55 billion How much genuine Arabic went in Paper, Table 3 Arabic after adding translated text … Read more