Khietsly Tristan's picture

6

Khietsly Tristan

khtsly

AI & ML interests

None yet

Recent Activity

commented on an article 2 days ago

The Optimal Architecture for Small Language Models

commented on an article 2 days ago

The Optimal Architecture for Small Language Models

new activity 2 days ago

codelion/dhara-70m:1024 in max_position_embeddings

View all activity

Organizations

None yet

commented on The Optimal Architecture for Small Language Models 2 days ago

also, pleasw consider to dclm-edu instead of dclm-baseline

commented on The Optimal Architecture for Small Language Models 2 days ago

the script is available in the model card.

New activity in codelion/dhara-70m 2 days ago

1024 in max_position_embeddings

#2 opened 2 days ago by

New activity in ibm-granite/GneissWeb 8 days ago

benchmark against Ultra-Fineweb?

#10 opened 8 days ago by

New activity in openbmb/Ultra-FineWeb 8 days ago

The dataset viewer is not available for this split.

#23 opened 8 days ago by

too much duplication in the dataset

#19 opened 5 months ago by

Does ultrafineweb_en_v1_4 overlap with ultrafineweb_en?

#22 opened 12 days ago by

New activity in HuggingFaceTB/finemath 4 months ago

random links in FineMath 4+

#51 opened 4 months ago by

commented on SmolLM3: smol, multilingual, long-context reasoner 4 months ago

it says beta1 0.8 in the picture but at somewhere of "Pretraining" it says "and the AdamW optimizer (beta1: 0.9, beta2: 0.95) with weight decay of 0.1"