2 1 6

George Cameron

georgewritescode

https://artificialanalysis.ai/

AI & ML interests

None yet

Recent Activity

updated a dataset about 8 hours ago

ArtificialAnalysis/AA-LCR

published a dataset about 11 hours ago

ArtificialAnalysis/AA-LCR

new activity about 11 hours ago

ArtificialAnalysis/AA-LCR:Upload AA-LCR_Dataset.csv

View all activity

Organizations

posted an update 6 days ago

Post

248

Announcing the Artificial Analysis Music Arena Leaderboard: with >5k votes, Suno v4.5 is the leading Music Generation model followed by Riffusion’s FUZZ-1.1 Pro.

Google’s Lyria 2 places third in our Instrumental leaderboard, and Udio’s v1.5 Allegro places third in our Vocals leaderboard.

The Instrumental Leaderboard is as follows:
🥇 Suno V4.5
🥈 Riffusion's FUZZ-1.1 Pro
🥉 Google's Lyria 2
- Udio v1.5 Allegro
- StabilityAI's Stable Audio 2.0
- Meta's MusicGen

Rankings are based on community votes across a diverse range of genres and prompts.

Participate in the arena and check out the space here:
ArtificialAnalysis/Music-Arena-Leaderboard

posted an update 12 days ago

Post

1042

🎵 Announcing Artificial Analysis Music Arena! Vote for songs generated by leading music models across genres from pop to metal to rock & more

Key details:
🏁 Participate in Music Arena and after a day of voting we’ll unveil the world’s first public ranking of AI music models.

✨ Currently featuring models from Suno, Riffusion, Meta, Google, udio and Stability AI!

🎤 Support for both a vocals mode and an instrumental mode

🎸 A diverse array of prompts from genres including pop, RnB, metal, rock, classical, jazz, and more

Check it out here:
ArtificialAnalysis/Music-Arena-Leaderboard

posted an update about 1 year ago

Post

1113

Visualization of GPT-4o breaking away from the quality & speed trade-off curve the LLMs have followed thus far ✂️

Key GPT-4o takeaways
‣ GPT-4o not only offers the highest quality, it also sits amongst the fastest LLMs
‣ For those with speed/latency-sensitive use cases, where previously Claude 3 Haiku or Mixtral 8x7b were leaders, GPT-4o is now a compelling option (though significantly more expensive)
‣ Previously Groq was the only provider to break from the curve using its own LPU chips. OpenAI has done it on Nvidia hardware (one can imagine the potential for GPT-4o on Groq)

👉 How did they do it? Will follow up with more analysis on this but potential approaches include a very large but sparse MoE model (similar to Snowflake's Arctic) and improvements in data quality (likely to have driven much of Llama 3's impressive quality relative to parameter count)

Notes: Throughput represents the median across providers over the last 14 days of measurements (8x per day)

Data is present on our HF leaderboard: ArtificialAnalysis/LLM-Performance-Leaderboard and graphs present on our website

1 reply

posted an update over 1 year ago

Post

2369

Excited to bring our benchmarking leaderboard of >100 LLM API endpoints to HF!

Speed and price are often just as important as quality when building applications with LLMs. We bring together all the data you need to consider all three when you need to pick a model and API provider.

Coverage:
‣ Quality (Index of evals, MMLU, Chatbot Arena, HumanEval, MT-Bench)
‣ Throughput (tokens/s: median, P5, P25, P75, P95)
‣ Latency (TTFT: median, P5, P25, P75, P95)
‣ Context window
‣ OpenAI library compatibility

Link to Space: ArtificialAnalysis/LLM-Performance-Leaderboard

Blog post: https://huggingface.co/blog/leaderboard-artificial-analysis

George Cameron

AI & ML interests

Recent Activity

Organizations

georgewritescode's activity