Post
1099
Two new entries to the AHA Leaderboard:
Kimi was OK until it started "thinking" ..
Kimi was OK until it started "thinking" ..
Emin Temiz PRO
etemiz
AI & ML interests
Alignment
Recent Activity
new activity
16 days ago
IslamQA/islamqa:Curation
posted
an
update
17 days ago
Two new entries to the AHA Leaderboard:
Kimi was OK until it started "thinking" ..
posted
an
update
18 days ago
Working on a new version of AHA Leaderboard.
- Better signal thanks to new models like Enoch
- MoA of top and bottoms of current leaderboard to add more diverse inputs
- Better questions
- A faster and more precise measurement methodology
- Explanations on how each column is calculated
- More sample questions revealed
Current version: https://huggingface.co/blog/etemiz/aha-leaderboard
Tell me how I can improve more.
Organizations
None yet
posted
an
update
17 days ago
posted
an
update
18 days ago
Post
1746
Working on a new version of AHA Leaderboard.
- Better signal thanks to new models like Enoch
- MoA of top and bottoms of current leaderboard to add more diverse inputs
- Better questions
- A faster and more precise measurement methodology
- Explanations on how each column is calculated
- More sample questions revealed
Current version: https://huggingface.co/blog/etemiz/aha-leaderboard
Tell me how I can improve more.
- Better signal thanks to new models like Enoch
- MoA of top and bottoms of current leaderboard to add more diverse inputs
- Better questions
- A faster and more precise measurement methodology
- Explanations on how each column is calculated
- More sample questions revealed
Current version: https://huggingface.co/blog/etemiz/aha-leaderboard
Tell me how I can improve more.
posted
an
update
about 1 month ago
Post
289
This is a very human aligned fine tune and would score 56 on AHA leaderboard:
CWClabs/CWC-Mistral-Nemo-12B-V2-q4_k_m
We are going to be using it as one of the ground truths for AHA Leaderboard 2.0 (the next version).
We will be able to generate some RL datasets for folks to align their own LLMs with humanity. We will generate answers from best models and worst models and do mixture of agents that combines the answer, and publish results as dataset(s). Things looking bright!
CWClabs/CWC-Mistral-Nemo-12B-V2-q4_k_m
We are going to be using it as one of the ground truths for AHA Leaderboard 2.0 (the next version).
We will be able to generate some RL datasets for folks to align their own LLMs with humanity. We will generate answers from best models and worst models and do mixture of agents that combines the answer, and publish results as dataset(s). Things looking bright!
posted
an
update
about 1 month ago
Post
299
Benchmarked 2 more models today. Any other model you want me to do?
Working on a broader version of the AHA leaderboard. Follow for more quackery :)
Working on a broader version of the AHA leaderboard. Follow for more quackery :)
replied to
their
post
about 2 months ago
This fine tuning would score 56 and be placed 1st in the leaderboard but I didn't add it, I only include full trainings in the leaderboard or (further tunings by the same company):
https://huggingface.co/CWClabs/CWC-Mistral-Nemo-12B-V2-q4_k_m
posted
an
update
about 2 months ago
Post
267
O oh! things looking uglier by day
LLM builders in general are not doing a great job of making human aligned models.
I don't want to say this is a proxy for p(doom)... But it could be if we are not careful.
Most probable cause is reckless training LLMs using outputs of other LLMs, and don't caring about curation of datasets and not asking 'what is beneficial for humans?'...
LLM builders in general are not doing a great job of making human aligned models.
I don't want to say this is a proxy for p(doom)... But it could be if we are not careful.
Most probable cause is reckless training LLMs using outputs of other LLMs, and don't caring about curation of datasets and not asking 'what is beneficial for humans?'...
-
- 1 reply
replied to
their
post
about 2 months ago
posted
an
update
about 2 months ago
Post
1065
Another abliteration by huihui which had big positive impact!
huihui-ai/Huihui-GLM-4.5-Air-abliterated-GGUF
@huihui-ai
huihui-ai/Huihui-GLM-4.5-Air-abliterated-GGUF
@huihui-ai
-
- 3 replies
replied to
their
post
2 months ago
Our leaderboard can be used for human alignment in an RL setting. Ask the same question to top models and worst models and the answer from top models can get +1 score, bad models can get -1. Ask many times with higher temperature to generate more answers. What do you think?
posted
an
update
2 months ago
Post
202
Grok 2 is worse than 1 but better than 3. This was already measured using API but now we measured the LLM and the results are similar.
GLM is ranking higher and higher compared to previous versions. Nice trend!
GLM is ranking higher and higher compared to previous versions. Nice trend!
-
- 1 reply
posted
an
update
2 months ago
Post
201
Benchmarked 5 models, and the winner of today is: Magistral !
https://huggingface.co/blog/etemiz/aha-leaderboard
https://huggingface.co/blog/etemiz/aha-leaderboard
posted
an
update
3 months ago
Post
252
https://x.com/OwainEvans_UK/status/1960359498515952039
It is weird that we can also understand humans more thanks to LLM research. Human behavior has parallels to this. When we optimize for short term pleasure (high time preference) we end up feeding the beast in us, ending up misaligned with other humans. But if we care about other humans (low space preference) we are more aligned with them. Feeding ego can have parallels to reward hacking. Overcoming ego can be described as having high human alignment score..
It is weird that we can also understand humans more thanks to LLM research. Human behavior has parallels to this. When we optimize for short term pleasure (high time preference) we end up feeding the beast in us, ending up misaligned with other humans. But if we care about other humans (low space preference) we are more aligned with them. Feeding ego can have parallels to reward hacking. Overcoming ego can be described as having high human alignment score..
posted
an
update
4 months ago
Post
2782
This sounds like people exchanging "vibes" when they are talking, sitting next to each other either with aromas or electromagnetic or scalar waves, or any other modes of exchange, maybe even including vaccine shedding :)
I think the reverse is also true, like a benevolent, properly aligned LLM can "subconscious teach" another LLM and proper alignment can spread like a virus.
https://x.com/OwainEvans_UK/status/1947689616016085210
I think the reverse is also true, like a benevolent, properly aligned LLM can "subconscious teach" another LLM and proper alignment can spread like a virus.
https://x.com/OwainEvans_UK/status/1947689616016085210
replied to
their
post
4 months ago
Hi Doctor Chad, nice to see you too
Thanks for sharing,
I will test that. Yi 1.5 has the second place on my leaderboard!
replied to
their
post
4 months ago
Enjoy!
If you were to change them to YES/NO or two-choice questions like
Is this bioactive compound beneficial to body or not?
Is this mycotoxin really a toxin and should be removed from foods?
Which mycotoxin is worse, A or B?
We could add them to the AHA leaderboard!
replied to
their
post
4 months ago
Interesting, I could test that as well.
Do you know their method of uncensoring? Are they fine tuning or doing vector operations?
I may upload a Qwen3 fine tune for AHA soon (would u like to merge others with it?).
posted
an
update
4 months ago
Post
325
I've tested many fine tunes. They were all getting lower scores than base in AHA.
Yesterday I found one fine tune (abliteration) which made the model go from 28 to 46: huihui-ai/Huihui-gpt-oss-120b-BF16-abliterated
Is there a correlation between censorship and being not human aligned?
Yesterday I found one fine tune (abliteration) which made the model go from 28 to 46: huihui-ai/Huihui-gpt-oss-120b-BF16-abliterated
Is there a correlation between censorship and being not human aligned?
-
-
- 5 replies
posted
an
update
4 months ago
Post
5692
benchmarked 9 models in 3 days. they were mostly below average in AHA score. p(doom) probably increased :(
replied to
their
post
4 months ago
maybe! i find LLMs to have not much integrity (not much correlation between domains) compared to a human.. a human can do interdisciplinary work better imo.
replied to
their
post
4 months ago