4 Commits

Author	SHA1	Message	Date
Yang Wang	65d8f5f1e9	Add CUDA cache clearing in memory management ... Added torch.cuda.empty_cache() to free up unused memory on the GPU,	2024-12-26 23:18:39 +08:00
Yang Wang	e6e66fd23f	sort filename to reduce memory costs	2024-12-26 23:14:44 +08:00
Yang Wang	1e3a83629e	handle missing scale_inv_name ... Fixed an issue where `weight` and `weight_scale_inv` (e.g. `model.layers.39.mlp.experts.92.gate_proj.weight` and `model.layers.39.mlp.experts.92.gate_proj.weight_scale_inv`) were not in the same SafeTensor, causing an assertion error due to scale_inv_name not being in the state_dict.	2024-12-26 23:09:17 +08:00
stack-heap-overflow	4c2fdb8f55	Release DeepSeek-V3	2024-12-26 19:01:57 +08:00