t5_dewata_reconstruct_task_mini

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0385

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 800
• eval_batch_size: 800
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 200
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.6946	1.0	1060	0.6088
0.6003	2.0	2120	0.5319
0.5445	3.0	3180	0.4737
0.494	4.0	4240	0.4256
0.4539	5.0	5300	0.3879
0.4157	6.0	6360	0.3507
0.3861	7.0	7420	0.3223
0.3635	8.0	8480	0.2975
0.3414	9.0	9540	0.2718
0.3165	10.0	10600	0.2525
0.2959	11.0	11660	0.2370
0.2757	12.0	12720	0.2188
0.2607	13.0	13780	0.2060
0.2449	14.0	14840	0.1932
0.2333	15.0	15900	0.1796
0.2188	16.0	16960	0.1703
0.2075	17.0	18020	0.1613
0.1944	18.0	19080	0.1517
0.1821	19.0	20140	0.1434
0.1737	20.0	21200	0.1356
0.1661	21.0	22260	0.1292
0.158	22.0	23320	0.1220
0.1486	23.0	24380	0.1142
0.1423	24.0	25440	0.1124
0.1326	25.0	26500	0.1046
0.1287	26.0	27560	0.1017
0.1204	27.0	28620	0.0949
0.1136	28.0	29680	0.0917
0.1089	29.0	30740	0.0874
0.1031	30.0	31800	0.0837
0.0978	31.0	32860	0.0798
0.0944	32.0	33920	0.0773
0.0903	33.0	34980	0.0751
0.0873	34.0	36040	0.0727
0.0816	35.0	37100	0.0699
0.0792	36.0	38160	0.0666
0.0747	37.0	39220	0.0661
0.0709	38.0	40280	0.0639
0.0681	39.0	41340	0.0616
0.0658	40.0	42400	0.0598
0.0624	41.0	43460	0.0588
0.0618	42.0	44520	0.0569
0.0587	43.0	45580	0.0559
0.0546	44.0	46640	0.0548
0.0537	45.0	47700	0.0537
0.0517	46.0	48760	0.0529
0.0492	47.0	49820	0.0525
0.0475	48.0	50880	0.0509
0.0471	49.0	51940	0.0517
0.0432	50.0	53000	0.0491
0.0434	51.0	54060	0.0486
0.0414	52.0	55120	0.0484
0.0402	53.0	56180	0.0486
0.0391	54.0	57240	0.0486
0.0369	55.0	58300	0.0469
0.0363	56.0	59360	0.0466
0.0351	57.0	60420	0.0472
0.0336	58.0	61480	0.0461
0.033	59.0	62540	0.0461
0.0314	60.0	63600	0.0453
0.0306	61.0	64660	0.0457
0.0306	62.0	65720	0.0456
0.0293	63.0	66780	0.0451
0.0282	64.0	67840	0.0444
0.0267	65.0	68900	0.0446
0.0274	66.0	69960	0.0441
0.0276	67.0	71020	0.0431
0.0259	68.0	72080	0.0438
0.0256	69.0	73140	0.0435
0.0238	70.0	74200	0.0438
0.0234	71.0	75260	0.0437
0.0226	72.0	76320	0.0437
0.0225	73.0	77380	0.0436
0.0222	74.0	78440	0.0437
0.0211	75.0	79500	0.0423
0.0206	76.0	80560	0.0425
0.0202	77.0	81620	0.0428
0.02	78.0	82680	0.0420
0.0189	79.0	83740	0.0428
0.0196	80.0	84800	0.0421
0.0183	81.0	85860	0.0422
0.0181	82.0	86920	0.0418
0.0181	83.0	87980	0.0415
0.0175	84.0	89040	0.0413
0.0171	85.0	90100	0.0419
0.0172	86.0	91160	0.0419
0.0166	87.0	92220	0.0411
0.0156	88.0	93280	0.0410
0.0162	89.0	94340	0.0421
0.0159	90.0	95400	0.0412
0.0155	91.0	96460	0.0410
0.0152	92.0	97520	0.0414
0.0149	93.0	98580	0.0411
0.0142	94.0	99640	0.0414
0.0144	95.0	100700	0.0403
0.0139	96.0	101760	0.0414
0.0138	97.0	102820	0.0407
0.0138	98.0	103880	0.0407
0.0131	99.0	104940	0.0400
0.0125	100.0	106000	0.0398
0.0122	101.0	107060	0.0406
0.0128	102.0	108120	0.0407
0.0111	103.0	109180	0.0404
0.0118	104.0	110240	0.0396
0.012	105.0	111300	0.0402
0.0115	106.0	112360	0.0398
0.011	107.0	113420	0.0407
0.0107	108.0	114480	0.0403
0.0108	109.0	115540	0.0408
0.0107	110.0	116600	0.0406
0.0104	111.0	117660	0.0404
0.01	112.0	118720	0.0404
0.0099	113.0	119780	0.0406
0.0099	114.0	120840	0.0402
0.0095	115.0	121900	0.0405
0.0094	116.0	122960	0.0397
0.0094	117.0	124020	0.0397
0.0092	118.0	125080	0.0393
0.0091	119.0	126140	0.0401
0.0088	120.0	127200	0.0397
0.0087	121.0	128260	0.0394
0.0087	122.0	129320	0.0394
0.0086	123.0	130380	0.0397
0.008	124.0	131440	0.0397
0.0083	125.0	132500	0.0397
0.0083	126.0	133560	0.0395
0.0082	127.0	134620	0.0394
0.0078	128.0	135680	0.0394
0.0075	129.0	136740	0.0385
0.0071	130.0	137800	0.0392
0.0075	131.0	138860	0.0389
0.007	132.0	139920	0.0392
0.0074	133.0	140980	0.0394
0.0071	134.0	142040	0.0391
0.0071	135.0	143100	0.0390
0.007	136.0	144160	0.0391
0.0069	137.0	145220	0.0397
0.0065	138.0	146280	0.0392
0.0061	139.0	147340	0.0395
0.0064	140.0	148400	0.0395
0.0058	141.0	149460	0.0402
0.0063	142.0	150520	0.0391
0.0058	143.0	151580	0.0391
0.006	144.0	152640	0.0396
0.006	145.0	153700	0.0392
0.006	146.0	154760	0.0394
0.0056	147.0	155820	0.0402
0.0058	148.0	156880	0.0395
0.0056	149.0	157940	0.0392
0.0055	150.0	159000	0.0395
0.0051	151.0	160060	0.0389
0.005	152.0	161120	0.0396
0.0053	153.0	162180	0.0394
0.0049	154.0	163240	0.0396
0.0048	155.0	164300	0.0391
0.0047	156.0	165360	0.0392
0.0047	157.0	166420	0.0389
0.0046	158.0	167480	0.0395
0.0048	159.0	168540	0.0388
0.0042	160.0	169600	0.0391
0.0043	161.0	170660	0.0393
0.0043	162.0	171720	0.0391
0.0042	163.0	172780	0.0389
0.0044	164.0	173840	0.0392
0.0041	165.0	174900	0.0386
0.004	166.0	175960	0.0388
0.004	167.0	177020	0.0387
0.0041	168.0	178080	0.0390
0.0038	169.0	179140	0.0387
0.0041	170.0	180200	0.0391
0.0039	171.0	181260	0.0384
0.0036	172.0	182320	0.0387
0.0034	173.0	183380	0.0388
0.0035	174.0	184440	0.0388
0.0035	175.0	185500	0.0389
0.0035	176.0	186560	0.0386
0.0033	177.0	187620	0.0387
0.0031	178.0	188680	0.0392
0.0032	179.0	189740	0.0386
0.0032	180.0	190800	0.0383
0.0033	181.0	191860	0.0385
0.0029	182.0	192920	0.0385
0.003	183.0	193980	0.0386
0.0033	184.0	195040	0.0385
0.003	185.0	196100	0.0384
0.0031	186.0	197160	0.0386
0.0029	187.0	198220	0.0385
0.0029	188.0	199280	0.0386
0.0028	189.0	200340	0.0386
0.0027	190.0	201400	0.0388
0.0028	191.0	202460	0.0387
0.0026	192.0	203520	0.0387
0.0025	193.0	204580	0.0387
0.0026	194.0	205640	0.0387
0.0027	195.0	206700	0.0386
0.0026	196.0	207760	0.0387
0.0025	197.0	208820	0.0386
0.0026	198.0	209880	0.0385
0.0027	199.0	210940	0.0384
0.0022	200.0	212000	0.0385

Framework versions

• Transformers 4.57.1
• Pytorch 2.9.0+cu126
• Datasets 4.0.0
• Tokenizers 0.22.1