AI software · Unified LLM gateway
TokenHub – Unified LLM Gateway
One entry point for every LLM. TokenHub connects your on-premises compute (AI Compute Appliances, your own GPUs) and public LLM APIs from major providers in one place. Requests go to local or public models automatically, based on your policies. With a RAG knowledge base, accounts and permissions, and usage and cost management built in, your company's AI use stays organized, controlled and accountable.
Key capabilities
Local and public models, one gateway
Register and call models on your on-premises appliances and GPU servers, and public LLM APIs from major providers, all through TokenHub. Mainstream Chinese and international model providers are supported.
Smart routing and fallback
Set routing rules by data sensitivity, cost, latency and model capability. Confidential data stays on local models. When local compute is busy or a task is not supported, requests switch to public models automatically, and fall back again if a public provider fails.
RAG knowledge base
Upload policies, manuals, contracts, tickets and other documents, and TokenHub splits, vectorizes and indexes them automatically. Answers cite their sources, and knowledge scope can be isolated by department. Common office document formats are supported.
Account and permission management
A three-tier account model for departments, users and applications. Assign models, knowledge bases and features by role, and sign in with your existing LDAP / WeCom / Feishu (Lark) accounts.
Usage and cost management
Set token quotas and budgets for each department, application and key. Local compute is costed by resources and public tokens by provider pricing, so you see in real time what you spend and what you save.
Audit and compliance
Every call is logged with the requester, model, and a summary of input and output. Sensitive-word and data-egress policies can work with the AI Firewall to meet internal audit requirements.
Where it fits
TokenHub sits between your applications and the models: one API upward; downward it manages both on-premises compute and public cloud models, and keeps knowledge bases, accounts, quotas and audit logs in one place.
Policies decide whether a request runs locally or in the public cloud
Admins configure routing rules in the console; no application code changes.
Sensitive data → local only
Requests marked as sensitive, such as contracts, customer data and R&D documents, always go to models on your on-premises appliance. Data never leaves your premises.
Local busy or not suited → switch to public
When the local GPU queue passes a threshold, or a task needs a larger model or multimodal capability, requests go to public LLMs automatically. Your applications see no difference.
Cost and quality → balanced by policy
Set “Cost first”, “Quality first” or “Speed first” policies per application, and keep backup public providers. If one provider rate-limits, another takes over automatically.
Modules
| Module | Features |
|---|---|
| Model gateway | OpenAI-compatible API, unified registration of local and public models, key custody, rate limiting and retries, streaming output |
| Routing engine | Routing by sensitivity / cost / latency / capability, failover, multi-provider backup, gradual rollout |
| RAG knowledge base | Document parsing and chunking, vector indexing, permission isolation, source citations, knowledge base Q&A and API |
| Account center | Three-tier accounts for departments / users / applications, role-based permissions, API key management, single sign-on |
| Usage and billing | Token quotas and budgets, statistics by department / application / model, cost comparison, report export |
| Audit and monitoring | Call logs, sensitive-operation auditing, model availability and latency monitoring, alerts |
Three deployment options
Preinstalled on the appliance
AI Compute Appliances ship with AI Inference Software and TokenHub preinstalled. Add your public provider keys on delivery and use local and public models side by side.
Install on your own servers
If you already have GPU or standard x86 servers, you can install TokenHub on its own, even just to manage public tokens. It runs on Linux servers with Docker; no GPU is needed if you only connect public models.
Hosted in HyperSource facilities
Deploy in HyperSource's Beijing, Guangzhou, Dongguan or Guiyang data centers, managed together with your appliance and inference software, and connected to your corporate network via private line or VPN.
Specifications
| Item | Details |
|---|---|
| Local model sources | AI Inference Software / vLLM / Ollama and other OpenAI-compatible services |
| Public model providers | APIs from mainstream Chinese and international LLM providers, added as needed |
| API | OpenAI-compatible REST API, SDK |
| Knowledge base | Common office document formats; capacity scales with deployment resources |
| Accounts and sign-in | Local accounts; LDAP / WeCom / Feishu (Lark) SSO |
| Deployment | Single node / high-availability cluster, Docker or bare-metal installation |
| Licensing | Preinstalled with the appliance, or per-node / annual subscription; subject to quotation |
Use cases
All of these can call models through TokenHub and share the same knowledge base.
Enterprise knowledge base Q&A
Turn policies, product materials and technical docs into an assistant you can ask directly.
Learn moreAI Kebao
AI customer service from HyperSource. AI answers first, human agents step in when needed, and every customer gets a follow-up. Inquiries from many channels flow into one service engine.
Learn moreIntelligent document processing
Extract and summarize key points from contracts, reports and forms, and cut repetitive reading and data entry.
Learn moreFAQ
I already have an AI Compute Appliance. Why do I need TokenHub?
The appliance gives you local compute. TokenHub governs how it is used: tasks that local models can't run or don't handle well can switch to public models, and every call has an account, quota and log, so you can see at a glance how much each department uses.
Where are public providers' API keys stored? Can business systems see them?
Provider keys are stored only in TokenHub. Business systems get internal keys issued by TokenHub, which can be revoked or capped at any time. Switching providers requires no changes to business code.
How does the RAG knowledge base relate to AI Kebao and the knowledge base Q&A app?
The TokenHub knowledge base is the underlying layer. AI Kebao, knowledge base Q&A and other apps reuse the same knowledge directly, with no duplicate uploads or upkeep.
We have no GPUs and only want to manage public LLMs centrally. Is that possible?
Yes. TokenHub can connect to public providers only. Bring accounts, quotas and costs under control first, and add local compute later.
How is it priced?
When preinstalled, it is included in the appliance price. Standalone deployments are licensed per node or by annual subscription, including upgrades and technical support; subject to quotation.
Related
Request a trial of TokenHub
A consultant will reply during business hours (China Standard Time) with site, configuration and pricing recommendations.
- leo@hs.com.cn
- 24/7 technical support