From 9693c71e523c774c465b42996c43ba0f2caf6835 Mon Sep 17 00:00:00 2001 From: "C.C." Date: Wed, 29 Jan 2025 21:16:26 +0800 Subject: [PATCH 1/3] docs: add yomo framework readme --- docs/yomo/README.md | 146 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 146 insertions(+) create mode 100644 docs/yomo/README.md diff --git a/docs/yomo/README.md b/docs/yomo/README.md new file mode 100644 index 0000000..1b727c1 --- /dev/null +++ b/docs/yomo/README.md @@ -0,0 +1,146 @@ +# YoMo Framework - Deepseek Provider + +YoMo is an open-source LLM Function Calling Framework for building Geo-distributed AI agents. Built atop QUIC Transport Protocol and Strongly-typed Stateful Serverless architecture, makes your AI agents low-latency, reliable, secure, and easy. + +## 🚀 Getting Started + +Let's implement a function calling serverless `sfn-get-ip-latency`: + +### Step 1. Install CLI + +```bash +curl -fsSL https://get.yomo.run | sh +``` + +### Step 2. Start the server + +Prepare the configuration as `my-agent.yaml` + +```yaml +name: ai-zipper +host: 0.0.0.0 +port: 9000 + +auth: + type: token + token: SECRET_TOKEN + +bridge: + ai: + server: + addr: 0.0.0.0:9000 ## Restful API endpoint + provider: deepseek ## LLM API Service we will use + + providers: + deepseek: + api_key: + model: deepseek-reasoner +``` + +Start the server: + +```sh +YOMO_LOG_LEVEL=debug yomo serve -c my-agent.yaml +``` + +### Step 3. Write the function + +First, let's define what this function do and how's the parameters required, these will be combined to prompt when invoking LLM. + +```golang +type Parameter struct { + Domain string `json:"domain" jsonschema:"description=Domain of the website,example=example.com"` +} + +func Description() string { + return `if user asks ip or network latency of a domain, you should return the result of the giving domain. try your best to dissect user expressions to infer the right domain names` +} + +func InputSchema() any { + return &Parameter{} +} +``` + +Create a Stateful Serverless Function to get the IP and Latency of a domain: + +```golang +func Handler(ctx serverless.Context) { + var msg Parameter + ctx.ReadLLMArguments(&msg) + + // get ip of the domain + ips, _ := net.LookupIP(msg.Domain) + + // get ip[0] ping latency + pinger, _ := ping.NewPinger(ips[0].String()) + pinger.Count = 3 + pinger.Run() + stats := pinger.Statistics() + + val := fmt.Sprintf("domain %s has ip %s with average latency %s", msg.Domain, ips[0], stats.AvgRtt) + ctx.WriteLLMResult(val) +} + +``` + +Finally, let's run it + +```bash +$ yomo run app.go + +time=2025-01-29T21:43:30.583+08:00 level=INFO msg="connected to zipper" component=StreamFunction sfn_id=B0ttNSEKLSgMjXidB11K1 sfn_name=fn-get-ip-from-domain zipper_addr=localhost:9000 +time=2025-01-29T21:43:30.584+08:00 level=INFO msg="register ai function success" component=StreamFunction sfn_id=B0ttNSEKLSgMjXidB11K1 sfn_name=fn-get-ip-from-domain zipper_addr=localhost:9000 name=fn-get-ip-from-domain tag=16 +``` + +### Done, let's have a try + +```sh +$ curl -i http://127.0.0.1:9000/v1/chat/completions -H "Content-Type: application/json" -d '{ + "messages": [ + { + "role": "system", + "content": "You are a test assistant." + }, + { + "role": "user", + "content": "Compare website speed between Nike and Puma" + } + ], + "stream": false +}' + +HTTP/1.1 200 OK +Content-Length: 944 +Connection: keep-alive +Content-Type: application/json +Date: Wed, 29 Jan 2025 13:30:14 GMT +Keep-Alive: timeout=4 +Proxy-Connection: keep-alive + +{ + "Content": "Based on the data provided for the domains nike.com and puma.com which include IP addresses and average latencies, we can infer the following about their website speeds: + - Nike.com has an IP address of 13.225.183.84 with an average latency of 65.568333 milliseconds. + - Puma.com has an IP address of 151.101.194.132 with an average latency of 54.563666 milliseconds. + + Comparing these latencies, Puma.com is faster than Nike.com as it has a lower average latency. + + Please be aware, however, that website speed can be influenced by many factors beyond latency, such as server processing time, content size, and delivery networks among others. To get a more comprehensive understanding of website speed, you would need to consider additional metrics and possibly conductreal-time speed tests.", + "FinishReason": "stop" +} +``` + +### Full Example Code + +[Full LLM Function Calling Codes](https://github.com/yomorun/llm-function-calling-examples) + +## 🎯 Focuses on Geo-distributed AI Inference Infra + +It’s no secret that today’s users want instant AI inference, every AI +application is more powerful when it response quickly. But, currently, when we +talk about `distribution`, it represents **distribution in data center**. The AI model is +far away from their users from all over the world. + +If an application can be deployed anywhere close to their end users, solve the +problem, this is **Geo-distributed System Architecture**: + +yomo geo-distributed system From 9f9414d4440ed3293ab0dbbc20e0ccd4142087de Mon Sep 17 00:00:00 2001 From: "C.C." Date: Wed, 29 Jan 2025 21:20:26 +0800 Subject: [PATCH 2/3] add yomo to readme --- README.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/README.md b/README.md index d781998..238b74c 100644 --- a/README.md +++ b/README.md @@ -148,6 +148,11 @@ English/[简体中文](https://github.com/deepseek-ai/awesome-deepseek-integrati Anda A Rust framework for AI agent development, designed to build a highly composable, autonomous, and perpetually memorizing network of AI agents. + + Icon + YoMo + Stateful Serverless LLM Function Calling Framework with Strongly-typed Language Support + ### RAG frameworks From 3607d68c364aa8b91f412663fd6c8f673976b1b5 Mon Sep 17 00:00:00 2001 From: "C.C." Date: Wed, 29 Jan 2025 21:23:20 +0800 Subject: [PATCH 3/3] add yomo to README_CN --- README_cn.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/README_cn.md b/README_cn.md index 567a9e9..0a603f2 100644 --- a/README_cn.md +++ b/README_cn.md @@ -122,6 +122,11 @@ Anda 一个专为 AI 智能体开发设计的 Rust 语言框架，致力于构建高度可组合、自主运行且具备永久记忆能力的 AI 智能体网络。 + + Icon + YoMo + Stateful Serverless LLM Function Calling Framework with Strongly-typed Language Support + ### RAG 框架