Merge pull request #123 from shreyaspimpalgaonkar/add-curator

feature: add Curator

README.md

@@ -178,6 +178,18 @@ English/[简体中文](https://github.com/deepseek-ai/awesome-deepseek-integrati
     </tr>
 </table>
 
+
+### Synthetic data curation
+
+<table>
+    <tr>
+        <td> <img src="https://raw.githubusercontent.com/bespokelabsai/curator/main/docs/Bespoke-Labs-Logomark-Red-crop.png" alt="Icon" width="64" height="auto" /> </td>
+        <td> <a href="https://github.com/deepseek-ai/awesome-deepseek-integration/blob/main/docs/curator/README.md"> Curator </a> </td>
+        <td> An open-source tool to curate large scale datasets for post-training LLMs. </td>
+    </tr>
+</table>
+
+
 ### IM Application Plugins
 
 <table>

docs/curator/README.md

@@ -0,0 +1,30 @@
+
+![image](https://raw.githubusercontent.com/bespokelabsai/curator/main/docs/Bespoke-Labs-Logomark-Red-crop.png)
+
+
+# [Curator](https://github.com/bespokelabsai/curator)
+
+
+Curator is an open-source tool to curate large scale datasets for post-training LLMs. 
+
+Curator was used to curate [Bespoke-Stratos-17k](https://huggingface.co/datasets/bespokelabs/Bespoke-Stratos-17k), a reasoning dataset to train a fully open reasoning model [Bespoke-Stratos](https://www.bespokelabs.ai/blog/bespoke-stratos-the-unreasonable-effectiveness-of-reasoning-distillation).
+
+
+### Curator supports:
+
+- Calling Deepseek API for scalable synthetic data curation
+- Easy structured data extraction
+- Caching and automatic recovery
+- Dataset visualization
+- Saving $$$ using batch mode
+
+### Call Deepseek API with Curator easily:
+
+![image](https://pbs.twimg.com/media/GiLHb-xasAAbs4m?format=jpg&name=4096x4096)
+
+# Get Started here
+
+- [Colab Example](https://colab.research.google.com/drive/1Z78ciwHIl_ytACzcrslNrZP2iwK05eIF?usp=sharing)
+- [Github Repo](https://github.com/bespokelabsai/curator)
+- [Documentation](https://docs.bespokelabs.ai/)
+- [Discord](https://discord.com/invite/KqpXvpzVBS)

docs/curator/README_cn.md

@@ -0,0 +1,29 @@
+![image](https://raw.githubusercontent.com/bespokelabsai/curator/main/docs/Bespoke-Labs-Logomark-Red-crop.png)
+
+
+# [Curator](https://github.com/bespokelabsai/curator)
+
+
+Curator 是一个用于后训练大型语言模型 (LLMs) 和结构化数据提取的制作与管理可扩展的数据集的开源工具。
+
+Curator 被用来制作 [Bespoke-Stratos-17k](https://huggingface.co/datasets/bespokelabs/Bespoke-Stratos-17k)，这是一个用于训练完全开源的推理模型 [Bespoke-Stratos](https://www.bespokelabs.ai/blog/bespoke-stratos-the-unreasonable-effectiveness-of-reasoning-distillation) 的推理数据集。
+
+
+### Curator 支持：
+
+- 调用 Deepseek API 进行可扩展的合成数据管理
+- 简便的结构化数据提取
+- 缓存和自动恢复
+- 数据集可视化
+- 使用批处理模式节省费用
+
+### 轻松使用 Curator 调用 Deepseek API：
+
+![image](https://pbs.twimg.com/media/GiLHb-xasAAbs4m?format=jpg&name=4096x4096)
+
+# 从这里开始
+
+- [Colab 示例](https://colab.research.google.com/drive/1Z78ciwHIl_ytACzcrslNrZP2iwK05eIF?usp=sharing)
+- [Github 仓库](https://github.com/bespokelabsai/curator)
+- [文档](https://docs.bespokelabs.ai/)
+- [Discord](https://discord.com/invite/KqpXvpzVBS)