gradio-chatbot
TypeScript icon, indicating that this package has built-in type declarations

0.0.28 • Public • Published

Gradio Chatbot

A tool that can automatically convert Huggingface Spaces, Modelscope Studios and Gradio ChatBot into free APIs. It basically supports any space with a chatbot, and currently perfectly supports many model spaces such as GPT4Free, ChatGPT, Llama 2, Vicuna, MPT-30B, Falcon, ChatGLM, Qwen and so on.

English | 中文

NPM Apache 2.0 License

Online Demo: https://weaigc.github.io/gradio-chatbot

Due to the current high demand on the ChatGPT space on Huggingface, there is a noticeable delay in response time. If you have your own ChatGPT account, it is recommended to use gpt-web.

Quick Start

NPM

  • Experience a free ChatGPT.
npx gradio-chatbot
# or
npm install -g gradio-chatbot
# call the default model
chatbot
  • Experience Llama2.
chatbot 2
# or
chatbot https://huggingface.co/spaces/huggingface-projects/llama-2-13b-chat

More usage just type chatbot help

Docker

docker build . -t gradio-server
docker run --rm -it -p 8000:8000 gradio-server

asciicast

Installation

You can use npm or yarn to install gradio-chatbot. Node version 18 or higher is required.

npm install gradio-chatbot
# or
yarn add gradio-chatbot

Usage

Currently supports three modes.

CLI mode

Refer to Quickstart.

API Service

To make it easy to use, two forms of interfaces are provided.

Streaming output, simply visit http://localhost:8000/api/conversation?model=0&text=hello. Non-streaming output, the calling method is the same as ChatGPT API. The following is an example of a call.

API Function

import { GradioChatBot } from 'gradio-chatbot'

const bot = new GradioChatBot();

async function start() {
  const message = await bot.chat('hello', {
    onMessage(partialMsg) {
      console.log('stream output:', partialMsg);
    }
  });
  console.log('message', message);
}

start();

You can also input the spatial address you want to convert, such as https://huggingface.co/spaces/h2oai/h2ogpt-chatbot.

import { GradioChatBot } from 'gradio-chatbot'

const bot = new GradioChatBot({
  url: 'https://huggingface.co/spaces/h2oai/h2ogpt-chatbot',
  fnIndex: 35,
}); // 调用自定义 ChatBot 模型

async function start() {
  console.log(await bot.chat('Hello'));
}

In addition, the NPM package has built-in support for 10 popular spaces from Hugging Face Spaces and Modelscope Studios. You can directly use the model index to access them. Please refer to the Model List for more details.

import { GradioChatBot } from 'gradio-chatbot';

const bot = new GradioChatBot('1');
async function start() {
  console.log(await bot.chat('Tell me about ravens.'));
}

start();

For more examples, please visit the directory: Examples .

Note: Some models on Hugging Face may collect the information you input. If you have data security concerns, it is recommended not to use them, and using self-hosted models is a better choice.

Using OpenAI

Python

import openai
openai.api_key = "dummy"
openai.api_base = "http://127.0.0.1:8080/v1"

# create a chat completion
chat_completion = openai.ChatCompletion.create(model="10", messages=[{"role": "user", "content": "Hello"}])

# print the completion
print(chat_completion.choices[0].message.content)

For more usage instructions, please refer to https://github.com/openai/openai-python

Node.js

import OpenAI from 'openai';

const openai = new OpenAI({
  baseURL: 'http://127.0.0.1:8080/v1'
});

async function main() {
  const stream = await openai.chat.completions.create({
    model: '10',
    messages: [{ role: 'user', content: 'Hello' }],
    stream: true,
  });
  for await (const part of stream) {
    process.stdout.write(part.choices[0]?.delta?.content || '');
  }
}

main();

For more usage instructions, please refer to https://github.com/openai/openai-node

API Document

See API Document

Model List

Index Type Description Model
0 Huggingface Spaces ChatGPT https://huggingface.co/spaces/yizhangliu/chatGPT
1 Huggingface Spaces GPT Free https://huggingface.co/spaces/justest/gpt4free
2 Huggingface Spaces Llama2 Spaces https://huggingface.co/spaces/ysharma/Explore_llamav2_with_TGI
3 Huggingface Spaces MosaicML MPT-30B-Chat https://huggingface.co/spaces/mosaicml/mpt-30b-chat
4 Huggingface Spaces Falcon Chat https://huggingface.co/spaces/HuggingFaceH4/falcon-chat
5 Huggingface Spaces Star Chat https://huggingface.co/spaces/HuggingFaceH4/starchat-playground
6 Huggingface Spaces ChatGLM2 https://huggingface.co/spaces/mikeee/chatglm2-6b-4bit
7 Huggingface Spaces ChatGLM https://huggingface.co/spaces/multimodalart/ChatGLM-6B
8 Huggingface Spaces Vicuna https://chat.lmsys.org/
9 Huggingface Spaces Qwen 7B Chat https://huggingface.co/spaces/mikeee/qwen-7b-chat
10 ModelScope Qwen 7B Chat https://modelscope.cn/studios/qwen/Qwen-7B-Chat-Demo/summary
11 ModelScope ChatGLM2 https://modelscope.cn/studios/AI-ModelScope/ChatGLM6B-unofficial/summary
12 ModelScope Jiang Ziya V1.1 https://modelscope.cn/studios/Fengshenbang/Ziya_LLaMA_13B_v1_online/summary
13 ModelScope Character Dialogue Chatbot developed by Alibaba DAMO Academy https://modelscope.cn/studios/damo/role_play_chat/summary

More useful models are welcome to contribute in the issue section.

Change Log

See CHANGELOG.md

Compatibility

  • This package supports node >= 18.

Credits

License

Apache 2.0 © LICENSE.

Package Sidebar

Install

npm i gradio-chatbot

Weekly Downloads

179

Version

0.0.28

License

Apache-2.0

Unpacked Size

605 kB

Total Files

40

Last publish

Collaborators

  • lwdgit